Re: thread heap leak?

[Carlos O'Donell <codonell at redhat dot com>]

On 4/30/19 5:48 PM, DJ Delorie wrote:
> David Muse <david.muse@firstworks.com> writes:
> > Ahh, ok.  I have a couple of questions then...
>
> Some of the answers are in the wiki:
> https://sourceware.org/glibc/wiki/MallocInternals
>
> > Is the 64MB segment size a result of my app doing 64MB of allocations,
> > or is that a default heap size?  Is that tuneable?

The first allocation of a new heap (attached to a given arena) creates
the virtual-address reservation of a size that matches the default
heap size, which is 64MiB for a 64-bit process.

This does not consume any memory though, it's just VA. RSS is used only
when you start dirtying pages.

> Tunables are documented in the manual, or in manual/tunables.texi in the
> source tree.  No tunables control the heap sizes, which are defined by
> the two constants HEAP_MIN_SIZE (32kb) and HEAP_MAX_SIZE (1Mb for 32-bit
> or 64Mb for 64-bit, IIRC)

And even if it was a tunable it would only change the increment of VA
reservation, and not the RSS used by the application e.g. you would
add X-sized heaps to the arena instead of Y-sized heaps.
 
> > Is there a way to inspect the base address of a thread's heap from
> > within the app?
>
> As per the wiki, the base address of the heap is just the pointer,
> masked to the right number of bits.  All heaps begin at a power of two.

Also, there isn't anything to inspect, the heap is an internal detail
of the implementation. You still haven't found the root cause of the
failure you're seeing.

> > If a thread exits, is its heap marked as available?  If so, is this
> > visible to the app?
>
> I don't think any heap-specific information is available to the app.

Correct. The heaps are an implementation detail. We really really should
have a heap dumper, a visualizer for the dump, and analysis tooling to
allow developers to see how the heap is interacting with their allocation
patterns... but we don't (yet).

> > At what point do new threads recycle old heaps?  Eg. Does a newly
> > created thread just grab a heap if an unused one is available, or is
> > there a minimum number of old heaps that have to pile up before they
> > will be recycled?  If there's a minimum number, is that tuneable?
>
> First off, don't confuse heaps with arenas.  There are a fixed max
> number of arenas, but each arena can have more than one heap as more
> memory is required.  Heaps may be returned to the kernel if all their
> memory is free'd, arenas are put on a free list when they're not used.
> New threads prefer to attach to an arena on the free list, else a new
> one is created if the max isn't reached, else arenas are shared.

Yeah, there is a terminology issue here, most developers think of their
application as having 2 things: a stack, and a heap. This isn't quite
the truth in a modern OS and is perhaps an oversimplification.

> > In a 64-bit app (on suse linux running on AWS (Xen)), should a
> > malloc() crash when the address space grows to 2G?
>
> No.

Agreed. You need to do a root cause analysis of this failure, perhaps
involving SUSE. However, Andreas (SUSE) commented upstream in this
issue thathe hasn't seen anything like this.
 
> (technicality: malloc() should never *crash* anyway, but it may return
> NULL and cause your application to crash ;)

David, I appreciate that you keep coming back to the list to discuss
this issue, and I find it very interesting to watch your progress. I hope
you get to the root cause!

--
Cheers,
Carlos.