Re: Getting systemtap examples working with --bpf backend

[William Cohen <wcohen at redhat dot com>]

On 5/16/19 9:41 AM, William Cohen wrote:
> I noticed https://elinux.org/images/d/dc/Kernel-Analysis-Using-eBPF-Daniel-Thompson-Linaro.pdf mentioned on page 29 that many of the systemtap examples did not work with the bpf back end and led to frustration.  Today I took a quick survey of how badly the examples are broken by adding the following line to the beginning of the run_command function in check.exp trying:
> 
>     set command [ string map {"stap " "stap --bpf "} $command ]
> 
> Most of the examples fail.  There are a few that actually do appear to run are just doing things in probe begin or end handlers (with the exception of cachestat*) :
> 
> PASS: systemtap.examples/general/ansi_colors run
> PASS: systemtap.examples/general/ansi_colors2 run
> PASS: systemtap.examples/general/helloworld run
> PASS: systemtap.examples/memory/cachestat run
> PASS: systemtap.examples/memory/cachestat_bpf run
> PASS: systemtap.examples/memory/kmalloc-top run
> 
> The non-bpf cachestat works because it just probes raw functions. Most examples fail because of various missing syscall.*/syscall_any probe points and gettimeofday_*() functions.  The time functions should be something easy to get working in bpf as there is already a nanosecond time function and its results could be scaled for microseconds, milliseconds, and seconds.
> 
> -Will
> 

Hi,

I have been looking at getting the syscall_any tapset working with
bpf.  If the syscall_any and syscall_any.return worked then the
following examples should work:

syscalls_by_pid.stp

There are other examples that use syscall_any and syscall_any.return,
but they have other issues like using multi-dimensional arrays, string
concentenation operations, or for loops that will prevent them from
working with the bpf backend.

I have made some modifications to provide the syscall_name and
syscall_num functions for bpf.  However, the code doesn't handle
32-bit compat syscalls properly.  This also gives warnings about
cross-file global variable references. This is on the
wcohen/bpf_syscall_any branch of systemtap git repo. Below is an
example running on x86_64:

$ ../install/bin/stap  --bpf -k -e 'probe oneshot {printf("%s\n", syscall_name(10))}'
WARNING: cross-file global variable reference to identifier '__syscall_32_num2name' at /home/wcohen/research/profiling/systemtap_write/install/share/systemtap/tapset/x86_64/syscall_num.stp:3:8 from: identifier '__syscall_32_num2name' at /home/wcohen/research/profiling/systemtap_write/install/share/systemtap/tapset/syscall_table.stp:8:16
 source:         return __syscall_32_num2name[num]
                        ^
WARNING: cross-file global variable reference to identifier '__syscall_64_num2name' at /home/wcohen/research/profiling/systemtap_write/install/share/systemtap/tapset/x86_64/syscall_num.stp:5:8 from: identifier '__syscall_64_num2name' at :11:12
 source:     return __syscall_64_num2name[num]
                    ^
WARNING: instance of overloaded function will never be reached: identifier 'syscall_name' at :5:10
 source: function syscall_name(num) {
                  ^
mprotect
Keeping temporary directory "/tmp/stapdkLJNc"


Taking a look at the syscall_any and syscall_any.return probes.  The
syscall_any can work in the existing bpf environment, but the
syscall_any.return uses some machine dependent C code possibly from
the kernel header to extract the syscall number the pt_regs. The
question is how this to keep it portable and avoid having the
debuginfo installed.  With the incomplete syscall_any tapset:

$ ../install/bin/stap  --bpf -k  testsuite/systemtap.examples/process/syscalls_by_pid.stp  -T 1
WARNING: cross-file global variable reference to identifier '__syscall_32_num2name' at /home/wcohen/research/profiling/systemtap_write/install/share/systemtap/tapset/x86_64/syscall_num.stp:3:8 from: identifier '__syscall_32_num2name' at /home/wcohen/research/profiling/systemtap_write/install/share/systemtap/tapset/syscall_table.stp:8:16
 source:         return __syscall_32_num2name[num]
                        ^
WARNING: cross-file global variable reference to identifier '__syscall_64_num2name' at /home/wcohen/research/profiling/systemtap_write/install/share/systemtap/tapset/x86_64/syscall_num.stp:5:8 from: identifier '__syscall_64_num2name' at :11:12
 source:     return __syscall_64_num2name[num]
                    ^
Collecting data... Type Ctrl-C to exit and display results
#SysCalls  PID
109        30875
117        27703
233        22764
317        20845
22         19624
1951       16890
219        14304
108        13864
548        13707
11         13665
110        13198
11         10789
11         10783
11         10780
11         10777
18         10520
1985       10372
17         10247
10         10014
19         9479
...

-Will