Re: RFE: dl_iterate_phdr, r_debug.r_brk usability for introspection

[OndÅej BÃlka <neleai at seznam dot cz>]

On Wed, Sep 16, 2015 at 12:21:51AM -0400, Rich Felker wrote:
> On Tue, Sep 15, 2015 at 08:54:40PM -0700, jreiser@bitwagon.com wrote:
> > On Tue, September 15, 2015 Rich Felker wrote:
> > > On Sun, Sep 13, 2015 at 09:45:22AM -0700, John Reiser wrote:
> > >
> > >> Counting and reporting the number of calls to dlopen and dlclose
> > >> would enhance the usability of dl_iterate_phdr.  A related change to
> > >> r_debug.r_brk would make life easier for live introspection.
> > >
> > > Can you offer some intended usage cases? I'm not seeing how this
> > > "enhances the usability".
> > 
> > Today it is cumbersome for an app to determine whether two invocations
> > of dl_iterate_phdr() generate the same list of loaded modules,
> > or to verify quickly the common cases when one is a subset of the other.
> > The app must test for equality of two sets.  This requires
> > storage whose size is unknown in advance, and some non-trivial code.
> > In contrast, if the number of calls to dlopen and dlclose are tracked
> > and reported, then in most common uses and environments the required storage
> > is two counters, and the code is trivial.  That's enhanced usability.
> > [Of course the worst case still requires general set difference;
> > but a very large fraction of practical cases do not.]
> > 
> > Suppose you are integrating code, shared libraries, and dynamic modules
> > from several sources.  The combination has a bug, and you want to locate it.
> > You identify some critical nodes and arcs in the high-level call graph,
> > and insert some introspecting code which calls dl_iterate_phdr.  The lists
> > of loaded modules at various times gives you significant information
> > about the state of the memory space, but making use of the information
> > is difficult because it is hard to compute the deltas between adjacent
> > samples.  But if the counts of dlopen and dlclose are tracked and reported,
> > then in most practical cases it becomes much easier.
> 
> Thanks for explaining.
> 
> > > The type unsigned is way too small for this and subject to overflow.
> > > Counters like this need to be at least 64-bit.
> > 
> > Claiming that more than 64 bits might be necessary is buffoonery; stop it.
> > In practice, overflowing a plain unsigned counter of the number of
> > dlopen or dlclose in a non-toy application takes a very long time.
> 
> I didn't claim that more than 64 bits are needed. I claimed that more
> than 32 bits are needed. It does take a while to overflow 32 bits, but
> not as long as you think (probably possible in ~1 month if you hammer
> it, and realistically on a scale of around a year), and low-quality
> interfaces that can overflow after a few weeks or months of uptime
> should not be added to glibc.
>
That is quite absurd that you need to wait that long. When you want to
find such issues you will debug/run application with added
dl_iterate_phdr and it wont take you so long to find problem.