Preparing for a new systemtap release?

[Mark Wielaard <mjw at redhat dot com>]

Hi,

It might be time to push out a new release again. We do publish regular
snapshots but people often just use the last released version. Seeing
that there have been various fixes for newer kernels and lots of new and
exciting stuff done since 0.8 maybe we should prepare for a 0.9 (or
0.8.1)?

I added some stuff to the NEWS file and prepared some draft release
notes below. Frank mentioned it would be nice to have a section on all
the kernel versions stap is know to work. I added my own, but you can
see I am pretty conservative. So what would be a good source for this.
Can dejazilla extract them?

I am not sure the process.mark(), sys/sdt.h and the dtrace compatible
static marker script are fully baked yet (I am still struggling with
getting all markers in hotspot visible myself). It seems important to
make sure at least the sys/sdt.h macros are as we really want before
recommending people try it out, so they don't need to unnecessary
recompile their binaries when we update these.

Are there any other known showstoppers?

And please add stuff in the blanks spots below or to the NEWS file if
you think I missed something (very likely there were 290 commits since
0.8).

Cheers,

Mark

Systemtap 0.9/0.8.1 (?) !!! DRAFT !!! release notes

= Systemtap frontend (stap) changes

  The stap "-r DIR" option may be used to identify a hand-made kernel
  build directory.  The tool determines the appropriate release string
  automatically from the directory.

  For scripts that sometimes terminate with excessive "skipped" probes,
  rerunning the script with "-t" (timing) will print more details about
  the skippage reasons.

  Per-pass verbosity control is available with the new "--vp {N}+"
  option.  "stap --vp 040" adds 4 units of -v verbosity only to pass
  2.  This is useful for diagnosing errors from one pass without
  excessive verbosity from others.

  For those that really want to run stap from the build tree there is
  now the 'run-stap' script in the top-level build directory that sets
  up the SYSTEMTAP_TAPSET, SYSTEMTAP_RUNTIME, SYSTEMTAP_STAPRUN, and
  SYSTEMTAP_STAPIO environment variables (installing systemtap, in a
  local prefix, is still recommended for common use).

For a full overview of the stap frontend see man stap(1).

= Systemtap client and compile server

  The systemtap client and compile server are now available.
  These allow you to compile a systemtap module on a host other than
  the one which it will be run, providing the client and server
  are compatible. Other than using a server for passes 1 through
  4, the client behaves like the 'stap' front end itself. This
  means, among other things, that the client will automatically
  load the resulting module on the local host unless -p[1234]
  was specified.

  The client/server now use SSL for network connection security and
  for signing.

  The systemtap client and server are prototypes only. Interfaces,
  options and usage may change at any time.

See stap-server(8) for more details.

= Systemtap probe points changes

  function("func").label("label") probes are now supported to allow
  matching the label of a function.

  Serious problems associated with user-space probing in shared
  libraries were corrected, making it now possible to experiment
  with probe shared libraries.  Assuming dwarf debugging information
  is installed, use this twist on the normal syntax:

    probe process("/lib64/libc-2.8.so").function("....") { ... }

  This would probe all threads that call into that library.  Running
  "stap -c CMD" or "stap -x PID" naturally restricts this to the target
  command+descendants only.  $$vars etc. may be used.

  process().mark() probes are now possible to trace static user space
  markers put in programs with the STAP_PROBE macro using the new
  sys/sdt.h include file. This also provides dtrace compatible markers
  through DTRACE_PROBE and an associated python 'dtrace' script that
  can be used in builds based on dtrace that need dtrace -h or -G
  functionality.

For a full overview of all probes see man stapprobes(5).

= Documentation

  SystemTap Tapset Reference Manual

  Standard tapsets included with Systemtap were modified to include
  extractable documentation information based on the kernel-doc
  infrastructure. When configured --enabled-docs a HTML and PDF
  version of the Tapset Reference Manual is produced explaining probes
  defined in each tapset.

  SystemTap Beginners Guide

  Systemtap now comes with a new Beginners Guide that walks the user
  through their first steps setting up stap, understanding how it all
  works, introduces some useful scripts and describes some common
  pitfalls.  It isn't created by default since it needs a Publican
  setup, but full build instructions can be found in the wiki:
  http://sourceware.org/systemtap/wiki/PublicanQuikHowto
  An online version can be found at:
  http://sourceware.org/systemtap/SystemTap_Beginners_Guide.pdf

= Miscellaneous changes

  Systemtap initscript is available. This initscript allows you to run
  systemtap scripts as system services (in flight recorder mode) and
  control those scripts individually.
  See README.initscript for details.

  Symbol tables and unwind (backtracing) data support were formerly
  compiled in for all probed modules as identified by the script
  (kernel; module("name"); process("file")) plus those listed by the
  stap "-d BINARY" option.  Now, this data is included only if the
  systemtap script uses tapset functions like probefunc() or
  backtrace() that require such information.  This shrinks the probe
  modules considerably for the rest.

  Most probe handlers now run with interrupts enabled, for improved
  system responsiveness and less probing overhead.  This may result in
  more skipped probes, for example if a reentrant probe handler is
  attempted from within an interrupt handler.  It may also make the
  systemtap overload detection facility more likely to be triggered,
  as interrupt handlers' run time would be included in the
  self-assessed overhead of running probe handlers.

= The following people helped with code contributions for this release:

  Dave Brolley, David Smith, Don Domingo, Elliott Baron, Eugeniy
  Meshcheryakov, Frank Ch. Eigler, Jim Keniston, Josh Stone, Kent
  Sebastian, K Prasad, Mark Wielaard, Masami Hiramatsu, Rajan Arora,
  Roland McGrath, Srikar Dronamraju, Stan Cox, Tim Moore, Wenji Huang
  and Will Cohen

= Range of kernel versions Systemtap has been tested on:

  2.6.27.12 (f10/i386/x86_64)
  2.6.18 (el5/xen/i386/x86_64)
  ...

= Know issues with this release:

...

= Bugs closed for this release:

...

= Test results on various systems:

...