Re: [PATCH] Fix tcache count maximum

[Wilco Dijkstra <Wilco dot Dijkstra at arm dot com>]

Hi Carlos,

> The tcache is a cost to the thread, above and beyond what it might be using,
> so in that sense we call it an "overhead." It's overhead because calling free
> will not lower the RSS used by the cache, nor will malloc_trim() reclaim it.

But in principle it could. It's an implementation issue, not a fundamental
design issue of per-thread caches.

> You make a statement about application workload patterns, could you expand on
> that a bit, I'd like to understand the conclusion you're trying to draw i.e.
> "few applications use".

Most applications only use a few different block sizes rather than every possible
size. You typically see a huge spike for just 2 or 3 unique sizes, and the rest is
in the noise. Hence the idea of caching blocks of the same sizes to improve
performance.

So it's misleading to quote the maximum amount of memory held in tcache as
a useful figure when no application would ever see that in reality. 

Note I see major speedups when increasing the tcache count. So we could simply
cache more small blocks as that is where the big gains are.

>> Secondly a single free block in tcache can block a whole multi-gigabyte arena
>> from being freed and returned to the system. That's a much more significant
>> bug than this maximum "overhead".
>
> This is a common complaint with heap-based allocators, and pathological worst
> cases can always be found for any allocator. It is a distinct issue from the
> issue at hand (though I'm happy to start another thread on the topic).

Sure but we're talking about blocks that have been freed, not blocks that are still in
use. The current implementation of tcache means that RSS size can be hugely
inflated compared to switching tcache off (far more than this "overhead"). The
current implementation cannot reclaim these freed blocks, but that could be fixed.

Anyway this is digression from the subject indeed, but the key point is that we can
improve the tcache significantly.

Wilco