Re: wrong calculation of kprobe.addr?

[Guang Lei Li <liguangl at cn dot ibm dot com>]

Hi,

  Sorry for the long reply. I suspect that the systemTap calculate the 
wrong address for kprobe to register a handler. It's maybe because of some 
optimization in GCC? I am not sure. Thanks in advance for your help.


------------------------------gcc version----------------------------
Reading specs from /usr/lib/gcc/i386-redhat-linux/3.4.4/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man 
--infodir=/usr/share/info --enable-shared --enable-threads=posix 
--disable-checking --with-system-zlib --enable-__cxa_atexit 
--disable-libunwind-exceptions --enable-java-awt=gtk 
--host=i386-redhat-linux
Thread model: posix
gcc version 3.4.4 20050721 (Red Hat 3.4.4-2)

--------------------root:/home/lgl> stap -v lgl.stp------------------
Created temporary directory "/tmp/stapitSQTW"
Searched '/usr/local/share/systemtap/tapset/2.6.14.2-ori/*.stp', match 
count 0
Searched '/usr/local/share/systemtap/tapset/2.6.14.2/*.stp', match count 0
Searched '/usr/local/share/systemtap/tapset/2.6/*.stp', match count 0
Searched '/usr/local/share/systemtap/tapset/*.stp', match count 9
Pass 1: parsed user script and 9 library script(s).
parsed 'elv_next_request' -> func 'elv_next_request'
pattern 'kernel' matches module 'kernel'
focused on module 'kernel' = [c0100000-c0464858, bias 0]
pattern 'elv_next_request' matches function 'elv_next_request'
selected function elv_next_request
querying prologue-end of function 'elv_next_request'
pattern 'kernel' matches module 'kernel'
Pass 2: analyzed user script.  1 probe(s), 6 function(s), 0 global(s).
Running grep " [tT] " /proc/kallsyms | sort -k 1,8 -s -o 
/tmp/stapitSQTW/symbols.sorted
Pass 3: translated to C into "/tmp/stapitSQTW/stap_2924.c"
Running make -C "/lib/modules/2.6.14.2-ori/build" M="/tmp/stapitSQTW" 
modules
make: Entering directory `/usr/src/linux-2.6.14.2'
  CC [M]  /tmp/stapitSQTW/stap_2924.o
  Building modules, stage 2.
  MODPOST
  CC      /tmp/stapitSQTW/stap_2924.mod.o
  LD [M]  /tmp/stapitSQTW/stap_2924.ko
make: Leaving directory `/usr/src/linux-2.6.14.2'
Pass 4: compiled into "stap_2924.ko"
Running sudo /usr/local/libexec/systemtap/stpd -r 
/tmp/stapitSQTW/stap_2924.ko
Running rm -rf /tmp/stapitSQTW
---------------------------------------------------------------------

I objdumped my kernel:
objdump -D vmlinux > dumpedFile

>From stap -p3:
static struct kprobe dwarf_kprobe_0[1]= {
  {.addr= (void *) 0xc02402f0}
};

>From /proc/kallsyms
  0xc02402e0 "elv_next_request"

>From dumpedFile
c02402e0 <elv_next_request>:
c02402e0:       push   %edi
c02402e1:       push   %esi
c02402e2:       mov    %eax,%esi
c02402e4:       push   %ebx
c02402e5:       sub    $0xc,%esp
c02402e8:       jmp    c0240309 <elv_next_request+0x29>
c02402ea:       lea    0x0(%esi),%esi
c02402f0:       orl    $0x2000,0x8(%ebx)
c02402f7:       mov    %edi,%ecx

You can see there is a jmp before c02402f0. But I don't know why not 
register hander at the address before c02402f0, such as c02402e4 or 
c02402e5?

--------------------------------------------
I also tried elv_remove_request and __elv_add_request because I CAN probe 
these two functions on the same machine on which elv_next_request failed 
to be probed.

>From /proc/kallsyms
0xc0240440 "elv_remove_request"

>From stap -p3:
static struct kprobe dwarf_kprobe_0[1]= {
  {.addr= (void *) 0xc0240441}
};

>From dumpedFile:
c0240440 <elv_remove_request>:
c0240440:       push   %ebx
c0240441:       mov    0x8(%edx),%ecx
c0240444:       mov    %eax,%ebx
c0240446:       mov    0xc(%eax),%eax
c0240449:       test   $0x40,%cl

-----------------------------------------------------------
>From  /proc/kallsyms:
  { 0xc02401d0, "__elv_add_request", "" },

>From stap -p3:
static struct kprobe dwarf_kprobe_0[1]= {
  {.addr= (void *) 0xc02401d6}
};

>From dumpedFile:
c02401d0 <__elv_add_request>:
c02401d0:       sub    $0x10,%esp
c02401d3:       mov    %ebx,(%esp)
c02401d6:       mov    0x14(%esp),%ebx
c02401da:       mov    %esi,0x4(%esp)
c02401de:       mov    %edx,%esi

-------------------------------------------------------------------
Finally I tried to use systemTap probe elv_next_request on my Power5, it 
worked for me.
I also dump the kernel in Power5, here's the result:

>From stap -p3:
static struct kprobe dwarf_kprobe_0[1]= {
  {.addr= (void *) 0xc0000000001dc81c}
};

>From /proc/kallsyms:
0xc0000000001dc7f8, ".elv_next_request"

>From dumpFile:
c0000000001dc7f8 <.elv_next_request>:
c0000000001dc7f8:       7c 08 02 a6     mflr    r0
c0000000001dc7fc:       fb 81 ff e0     std     r28,-32(r1)
c0000000001dc800:       fb c1 ff f0     std     r30,-16(r1)
c0000000001dc804:       fb a1 ff e8     std     r29,-24(r1)
c0000000001dc808:       fb e1 ff f8     std     r31,-8(r1)
c0000000001dc80c:       eb c2 dc a0     ld      r30,-9056(r2)
c0000000001dc810:       7c 7c 1b 78     mr      r28,r3
c0000000001dc814:       f8 01 00 10     std     r0,16(r1)
c0000000001dc818:       f8 21 ff 71     stdu    r1,-144(r1)
c0000000001dc81c:       48 00 01 2c     b       c0000000001dc948 
<.elv_next_request+0x150>

I attached the dumpedFile:

Attachment: dumpedFile.tgz
Description: Binary data