Re: [RFC] nptl: change default stack guard size of threads

[Szabolcs Nagy <nsz at port70 dot net>]

* Florian Weimer <fweimer@redhat.com> [2017-11-29 21:44:14 +0100]:
> On 11/29/2017 07:40 PM, Szabolcs Nagy wrote:
> > On 29/11/17 15:18, Florian Weimer wrote:
> > > On 11/29/2017 03:59 PM, Szabolcs Nagy wrote:
> > > > The change can be made for aarch64 only
> > > 
> > > That doesn't seem to be the case, looking at the patch.
> > > 
> > > So what you intended to do, exactly?
> > 
> > it seems the issue applies to all targets, and since
> > glibc itself have 64k stack jumps i sent the rfc
> > patch for all targets to err on the safe side.
> 
> glibc has many arbitrarily large stack jumps (although we've been
> eliminating them manually for a while).
> 
> What should guide the default size of the guard is not what glibc needs for
> its own routines, but what the stack probing in GCC needs to be correct.
> 

assuming you dont care about legacy binaries or binaries
where probing was turned off.

> > some targets will have 4k probe interval in gcc but
> > even those are not safe with existing binaries and
> > several targets have no proposed stack probe patch
> > as far as i understand.
> 
> Isn't there are generic variant which covers at least alloca and VLAs?
> 
> > > A 64 KiB probe interval on legacy 32-bit architectures is really a no-go.  It means we have to increase the
> > > guard region size to 64 KiB. But we cannot do that: The guard allocation comes out of the overall thread stack
> > > size, and existing applications do not expect that 60K of configured stack suddenly becomes unavailable.
> > > Adding the guard size on top of the allocation will break setups which are carefully tuned for a maximum number
> > > of threads.
> > 
> > i was aware of the address space limitation on 32bit
> > but e.g. aarch64 ilp32 will need the 64k guardsize too.
> 
> Why?
> 
> This is a new feature.  Why make this less usable from the start?
> 
> (I don't care about aarc64 ILP32 and page sizes smaller than 64 KiB on
> aarch64 in general, so I wont argue this, and this is just a courtesy
> notification that what you are doing is Very Wrong Indeed.)
> 
> > (i think there are other 32bit targets that support
> > > 4k page size, those may not mind the increase either,
> > they have to portably support large page size anyway)
> 
> GCC needs to emit probe intervals for the smallest supported page size on
> the the target architecture.  If it does not do that, we end up in trouble
> on the glibc side.
> 
> We can throw new code at this problem and solve it for 64-bit.  For 32-bit,
> we simply do not have a universally applicable solution.  My understanding
> was that everywhere except on ARM, GCC was compatible with the pioneering
> glibc/Linux work in this area (the guard page we added to thread stacks, and
> the guard page added by the kernel).  If this isn't the case, then I'm
> really disappointed in the disregard of existing practice on the GCC side.

i was not involved in the gcc probing design, but i noticed
the glibc implications that's why i posted a patch, my
understanding is that the efficiency of the probing on a
target depends on

- is there an implicit probe already at calls? (return
address stored on stack vs in register) if not then a
function using the stack must make an assumption about
how far the previous probe might have been (this is a
new call abi)
- is there a way to atomically increment sp and probe it
or access memory below sp?  if not then sp can only be
incremented by guardsize-eps at a time where eps depends
on how signal frame is stored. (and if signal frame is
large and written the wrong way then it may be impossible
to avoid memory writes accross a guard at all times)
- frame layout (incomming/outgoing args, locals, frame
pointer etc) can make probing require significant changes
to function prologues (extra instructinos) or not.

on targets like aarch64 the most conservative design
might have enough performance impact to make users turn
it off.

ilp32 has the same probing design as lp64 so it requires
the same guardsize.

to me it seemed a tradeoff between performance overhead
vs address-space overhead, if there are runtime troubles
because of the larger guardsize requirement then the
tradeoff may need to be revisited.
(with better data on both sides)

i also didnt think 64k guard would be prohibitive on
ilp32 (i know there are usecases where it is a problem,
but i expected those to be very rare cases)