Re: [patch] malloc per-thread cache ready for review

[Carlos O'Donell <carlos at redhat dot com>]

On 02/01/2017 11:54 AM, Markus Trippelsdorf wrote:
> On 2017.02.01 at 11:44 -0500, DJ Delorie wrote:
>>
>> Markus Trippelsdorf <markus@trippelsdorf.de> writes:
>>> http://locklessinc.com/downloads/lockless_allocator_src.tgz (the best in
>>> my testing) or jemalloc.
>>
>> Before we go down the "which allocator is best" road... glibc's
>> allocator is intended to be a general purpose "reasonably good enough"
>> system allocator.  It's easy to find a specific allocator that beats it
>> in a specific test, but being a specifically best allocator is not our
>> goal here - providing an allocator that can be the default on a
>> Linux-based system is.
>>
>> Hence, my goal with the per-thread cache is to make it "generally
>> better" for overall system performance.
>>
>> I am not trying to make it better than every other allocator in every
>> case, that's a futile exercise.
> 
> Well, there wouldn't be a reason for all these alternative allocators if
> glibc's would be "reasonably good". In fact is often astonishingly bad.

Given apriori knowledge of the workload allows you choose an allocator 
whose semantics match your allocation pattern and that improves
performance, and memory usage.

There are many more allocators than tcmalloc and jemalloc, and many more
embedded allocators in projects that you don't readily have visibility
into.

For a general purpose allocator the performance of the allocator can 
only be measured against a given corpus of workloads.

To this day no serious corpus of workloads has been collected to measure
allocators against. All the academic papers I've seen only test against
a few workloads.

I hope that within glibc we can gather up workloads to test the allocator
and raise the performance and quality. We have started gathering malloc
traces for just this purpose.

Regarding your comments about glibc malloc being astonishingly bad, do
you have a reference to such a workload? I am looking for _real_
workloads not synthetic ones created to show worst case behaviour in
heap-based allocators (dlmalloc, ptmalloc, and glibc's malloc).

> Examples are all major browsers (using jemalloc or tcmalloc) and Rust
> (jemalloc gets linkend in for all generated binaries by default).

TLDR; I don't think any of these examples chose jemalloc because glibc's
malloc was bad, but because they wanted to offer a choice, and fix
Windows flaws.

I can only speculate here because few projects provide detailed analysis
backed up by real data as the rationale for switching to an alternate
allocator.

Firstly the browsers were looking for a cross-OS solution to solving
memory fragmentation issues in Windows, something we don't specifically
cater to in glibc, but which the portable jemalloc did solve. This was
in the FF3 era when jemalloc was added and it solved the Windows XP
fragmentation issues.

The major browsers use forks of jemalloc. As far as I can tell the forks
are no longer distinguishable from the originals after their modifications
e.g. jemalloc vs. mozjemalloc. Though they do merge in jemalloc enhancements.

I have heard that Firefox needed fast fixed-size caches, and jemalloc was
a suitably licensed and flexible framework to build that with (glibc
was not). Similarly jemalloc is bundled as a permissively licensed
allocator upon which to build your own allocator, and it has become the
defacto standard for that.

Also note that Intel has memkind intended for a NUMA-aware API and that
is built around jemalloc. Here the discussion is much more about a modern
allocator API that supports future NUMA systems. The API itself if well
defined could be implemented in glibc's malloc IMO.

Rust has a concept of "custom allocators" so they didn't choose jemalloc
for any particular reason I can see, just that they acknowledge that
'system_malloc' might not meet your application needs and so provide
alternate allocators, including jemalloc, though Rust could add any
other allocators, and so can users via the custom allocators interface.

Again, nobody was saying glibc malloc was terrible in either of these
cases.

I have nothing but great esteem for the jemalloc and tcmalloc developers,
the allocators are brilliant and very good, and I think that perhaps
page-based allocators may be the better solution for a general purpose
allocator. However, I want to gather the requisite workloads to show
that, and make an informed decision for glibc.

-- 
Cheers,
Carlos.