[Bug runtime/17256] New: investigate on-the-fly support for other probe types

["jlebon at redhat dot com" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=17256

            Bug ID: 17256
           Summary: investigate on-the-fly support for other probe types
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: runtime
          Assignee: systemtap at sourceware dot org
          Reporter: jlebon at redhat dot com

As a follow-up to bug 10995, investigate other probe types where support for
on-the-fly operations would be worth adding (e.g. probe types with high
overhead cost).

BASIC IMPLEMENTATION INFO

When a probe type supports on-the-fly operations, it means that it is capable
of being turned off (or at least put in a lower-overhead state) during runtime
depending on the value of the probe condition.

For example, imagine we have:

probe foo if (bar) { ... }

If the underlying probe type of 'foo' has otf support, then it will be turned
off (or on) whenever the expression 'bar' transitions from true to false (or
false to true).

The way it works underneath is that any probe which writes to a global var
involved in the evaluation of 'bar' also sets a flag (need_module_refresh) if
the evaluation result has changed. The current value of the evaluation is
stored in the 'cond_enabled' field of the affected probe.

Upon exiting the probe, if the context is safe enough, schedule_work() is
called so that a work item is added to the system queue (module_refresh_work).
This work will eventually call systemtap_module_refresh() in process context,
at which point the affected probes are turned on/off as dictated by the
'cond_enabled' fields.

If the context is not safe for workqueue manipulation, then schedule_work() is
not called. Instead, an always-running background timer (module_refresh_timer)
eventually fires, which does the call to schedule_work() if necessary.

ADDING SUPPORT FOR NEW PROBE TYPES

Here are roughly the necessary steps for adding support:

1. Override the otf_supported() and otf_safe_context() functions of the
specific derived_probe_group:

- otf_supported() should return true if on-the-fly is supported. This will
cause the need_module_refresh flag to be set whenever probes of that type have
their conditions changed, which will eventually cause
systemtap_module_refresh() to be called.
- otf_safe_context() should return true if calling schedule_work() is safe to
do in the context of that probe type. If not, then return true (the background
timer will automatically be set to run). In general, hardirq context is safe,
while softirq may not be. Probes which fire in process context are most likely
safe as well.

Note how otf_supported() and otf_safe_context() are independent of each other.
The former determines whether the probe type can be turned on/off, while the
latter merely indicates whether calling schedule_work() is a safe thing to do.

Although otf_safe_context() currently defaults to 'false' for mostly all probe
types, it may be worth it to go over all the probe types and override it to
'true' wherever appropriate. This will allow us to reduce dependence on the
background timer.

2. Write the actual refresh code. You will need to modify the code in
emit_module_refresh() for that derived_probe_group (or the code in the function
which is called by the code emitted in emit_module_refresh()). In that code,
inspect the values of cond_enabled for the probes in your group and turn on/off
as needed.

For references, see the following exmaples:
- stapkp_refresh() in runtime/linux/kprobes.c, 
- stapiu_refresh() in runtime/linux/uprobes-inode.c
- hrtimer_derived_probe_group::emit_module_refresh() in tapset-timers.cxx

You might need to create a new field in the probe type specific struct to track
whether the probe is enabled or disabled. If you can directly query the kernel
that's even better (e.g. for kprobes, we have kprobe_disabled(), while for
inode-uprobes, we had to add a 'registered' field to stapiu_consumer).

Use dbug_otf() whenever you turn on/off probes. This will be useful for
testing.

3. Write the testcase. The testsuite/systemtap.onthefly directory is where the
new testcase would be added. Look at the already existing examples there
(hrtimer, kprobes, and uprobes_onthefly.exp) and go from there. The idea is
both to verify correct turning on and off in various situations, as well as
stress-testing it.

4. Profit

And that's about it! Again, for reference, you can always look back at how otf
support was added to kprobes, inode-uprobes, and hrtimer probes.

-- 
You are receiving this mail because:
You are the assignee for the bug.