Re: [PATCH] Count number of logical processors sharing L2 cache

["H.J. Lu" <hjl dot tools at gmail dot com>]

On Tue, May 24, 2016 at 2:35 PM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Tue, May 24, 2016 at 10:49 AM, Carlos O'Donell <carlos@redhat.com> wrote:
>> On 05/24/2016 11:02 AM, H.J. Lu wrote:
>>> CAT applies to a specific thread/process. Cache sizes in glibc are applied
>>> to string/memory functions for all threads/processes.  They both try to avoid
>>> over-using shared cache by a single thread/process.  But they work at
>>> different levels and have different behaviors.  Glibc also uses the cache size
>>> to decide when to use non-temporal store to avoid cache pollution and speed
>>> up writing a large amount of data..
>>
>> Don't you mean that CAT applies to a core (and all of its logical cores)?
>>
>> Might it be the case that a thread or process could be migrated by the linux kernel
>> between various cores configured with different CAT values and the glibc heuristics
>> could be poorly tuned for some of those cores?
>>
>> As I see it the values computed by init_cacheinfo() are only average heuristics for
>> the core.
>>
>> I agree that Florian has a point, that these values may become less useful in the
>> presence of the dynamically changing L3<->core partitioning enabled by CAT.
>>
>> It is silly though to think that you would allow a thread or process to migrate
>> away from the CAT-tuned core. The design of CAT is such that you want to isolate
>> the tuned application to one ore more cores and use CAT to control the L3 allocation
>> for those cores.
>
> I checked with our kernel CAT implementer.  CAT supports both
> processor and process.
>
>> In the case where you have a process pinned to a core, and CAT is used to limit the
>> L3 of that core, do the glibc heuristics computed in init_cacheinfo() match the
>> reality of the L3<->core allocation? Or would a lower L3 CAT-tuned value mean that
>> glibc would be mis-tuned for that core?
>
> CAT dedicates part of L3 cache to certain processor or process so
> that L3 cache is always available to them.  Glibc tries not to take all
> L3 cache in memcpy/memset so thar L3 cache is available for other
> operations within the same process as well as to other processor/process.
> CAT and glibc work at different angels.  There is no direct conflict between
> CAT and glibc.  At the moment,  I am not sure if CAT-aware glibc will
> improve performance.

I will check in my patch shortly.

-- 
H.J.