Re: user-space probes -- plan B from outer space

[Vara Prasad <prasadav at us dot ibm dot com>]

Frank Ch. Eigler wrote:

> prasadav wrote:
>
>  
>
> > [...]  Well another approach is similar to Ptrace actions you have
> > predefined handlers that some one can activate through a new
> > systemcall. [...] 
> >    
>
> A fixed pool of predefined handlers seem like an antithesis of
> systemtap.  Did you have some user interface in mind for these?
>  
Like i mentioned in response to Prasanna's user space probes approach  
mail  there will be two types of handlers for user space probes. One 
type is predefined for specific tasks like dump the stack, get registers 
etc., these handlers are built into the kernel. The second type of 
handlers are the ones that can be added on demand using a module load 
and register them similar to kprobes. SystemTap will use the second 
approach.

Association of a break point to handler is done as a separate step and 
either type of the handlers can be associated with the breakpoint. The 
idea of predefined handlers is if you don't want to write any 
complicated handlers and probe only your own process you could use the 
preexisting handlers and don't need to write kernel module and possibly 
don't even need root permission to trace.  This addresses one of the 
comments we got for earlier userspace implementation posting in LKML.

>  
>
> > >  user.process(233).statement(0xfeedface)
> > >  user("fche").process("/bin/vi").function("*init*")
> > >      
> > In my opinion user space probes is most useful in the case of server
> > class kind of complicated programs which are usually long living,
> > [...]  there is not much value with system wide tracing instead we
> > should focus on process specific tracing. [...]
> >    
>
> There is no contradiction here.  System-wide probing can be
> accomplished by a collection of process-specific probes.
>
>  
>
> > [...] I am not sure i see the value of process("process name")
> > syntax if our focus is process specific tracing.
> >    
>
> It would be one way of identifying present or future processes to
> probe.  For processes that do not yet exist, what other scheme do you
> have in mind?
>  
The scheme i am thinking is you could start a new process using 
systemtap, systemtap will then fork a copy of itself and exec the new 
executable on the child. This gives systemtap ability to put probes on 
the child process from the start similar to what gdb does if you start a 
program under debugger.

With the ability to probe already running process and as well as able to 
start new  process under systemtap we can probe most applications.

The one case that we can not cover with the above two is if an 
application has a complicated startup. For example if script A starts 
Script B and script B starts an executable C and executable C is a short 
lived one, since C is a short lived one it will not give us a chance to 
put probes before it exits. I think these kinds of cases are not too 
many to worry for now at least. Do you think of any other useful cases 
that we need to probe not covered by the above two approaches?

>  
>
> > I am not sure i see lot of value of this solution compared to a gdb
> > batch job, but for bit better performance than the heavy weight gdb.
> > [...]
> >    
>
> How would this gdb batch job alternative work?  Are you intending to
> compare the expressity of systemtap script with gdb macros?
>
>  
I am not saying gdb macros are as powerful as systemtap scripts. All i 
meant is one can use gdb batch scripts to print the variables you need 
and use all kinds of post processing using your favorite scripting 
language. As the handlers are run in the userspace i am not seeing much 
advantage of  using systemtap language to do filtering in the userspace 
vs post processing using your favorite scripting language.

[...]

> - FChE
>