Re: systemtap/pcp integration

["Frank Ch. Eigler" <fche at redhat dot com>]

Hi -

dsmith wrote:

> [...]
> > Overall, are you happy with the general approach of reusing the exact
> > MMV format (and thus the PMDA)?
> 
> [...]
> However, as I've worked with the MMV format I've come to realize its
> limitations. As Nathan has pointed out in another email, the MMV format
> is designed to only support exporting values, and isn't suited for more
> event-like tracing. As far as the more technical side of things goes,
> some of the internal offset logic might be done better/differently.

An application of pmda/mmv & pmda/logger to the same stap module could
perhaps accomplish both goals (assuming we consider the pcp events
overengineered to the extent that supplying timestamped strings is
sufficient).  Have you considered an alternative unified design?

This reminds me of another PCP PMDA we've mentioned in the past: a
JSON fetcher/parser.  We'll need something like this for a variety of
non-systemtap purposes too (interop with JSON-producing tools).  What
if stap were to produce pcp metrics in the form of /proc/systemtap/*
JSON files that the PMDA would read on demand?  (The cost of the
parsing overhead may be low enough not to worry about it.)  A separate
generated JSON file could provide metadata.  That format could be rich
enough to contain events too (mapped from arrays of string).


> > At one point I suggested reworking the earlier prototype so that the
> > bulk of the MMV format's emulation would be based on tapset script
> > code (and possibly more declarative / dynamic / safe) rather than C.
> > Have you come to any conclusions about the propriety of that?
> 
> I've been focused on other things, like reworking the allocation logic.
> As you describe it above, I'm not sure I see where you are headed.

To spell it out, the idea was to encode the mmv format logic
(including metadata management) within a stap tapset script instead of
as C in the runtime.  Then the C runtime would need to do nothing but
provide a memory-mapped-byte-array kind of abstraction, and a way for
the script code to read/write it (maybe a variant of
sprintf("%b...")?).


> [...]
> > How much post-initialization change can the MMV format tolerate, as
> > regarding indom contents or metric availability?  I assume such
> > metadata changes are synchronized with the PMDA via the generation
> > numbers.  Moving around contents of the mmap region as in
> > __stp_mmv_alloc_data_item sounds like it leaves the data inconsistent
> > during the process; does it need similar protection?
> 
> The MMV format doesn't support any post-initialization changes - once
> you call "start" the file format can't change without removing and
> recreating the file. (The reader knows "start" has been called based on
> when the generation numbers match.) [...]

(I think generation numbers can be changed during the run, to trigger
a pmda/mmv reload, but don't know how thoroughly that works.)


- FChE