Re: What about MIPS support?

[Victor Kamensky <kamensky at cisco dot com>]

Hi All,

It took us much much longer than originally expected. But it is better 
late than never. Here is our patches tarbal attached. Inside of tarbal 
(README file) and copied below I put short description of each patch. Some 
of patches are Cisco specific and may not be any interest for you. Some 
like MIPS support, cross compilation improvements, misc general fixes (i.e 
systemtap issue 6977), I think, could be quite useful to community. Since 
it is quilt patch series and patches may have patch apply order issues, we 
are publishing all of them, exactly as we use/apply them, regardless 
whether we consider them Cisco specific or not.

As it was discussed, for now, I am just dropping our patches as is. Latter 
as time permits, I can work on cleaning them up, moving to latest 
systemtap git and try to prepare them for commit into systemtap tree. It 
would be great if folks could take a look at patches and/or patches 
description and indicate priorities/interests for specific patches on 
which I could start working first.

Please advise if you would like me to post the same on different subject 
line on this mailing list (since it covers more than just mips support). I 
thought I just post it on original thread to get it proper closure.

==== systemtap-1.6-cisco-patches/README start =============================

This is patches series that Cisco did on top of systemtap-1.6. Some of them are 
Cisco specific and most likely are not interesting for community. We decided to
provide them here anyway in order to avoid breakage in applying patches, because
of patch application order issues.

Other like MIPS support, cross compilation improvements, misc general fixes may
be some interest for SystemTap community.

Patch description follows order of patches in series file:

systemtap-configure_cross_compile.patch - change in configure logic to make it
    more cross compilation environment friendly (in cross compile environment
    one cannot test presence of /usr/include files; those are from host). We
    don't use nss3 and avahi in our target so for now just introduce option
    that could disable them. Most likely issue is fixed in latest tree.

systemtap-unwind-table-size.patch - IOS code is too big, we are hitting limit
    on unwind symbol table size. Increase the limit for now. In future it would
    be good code to adjust those values dynamically.

systemtap-fde_byte_order_fix.patch - Fix reading fde entry in case of cross
    compilation where target and host have different byte order (i.e x86 host,
    be mips is target).

systemtap-pointer-byte_size.patch - Fix related to ICC (intel compiler). There
    is a difference how DW_AT_byte_size is stored between gcc and icc. Current
    stap code tailored to gcc way. Need to go back and dig out more details
    here.

systemtap-userland_caller.patch - Trying to introduce ucaller_addr and ucaller
    systemtap function. Most likely this code does not work. But currently
    patch is in our series file so keeping it for now.

systemtap-biendian.patch - Support ICC biendian feature. It is described at
    http://software.intel.com/en-us/articles/dwarf-extensions-for-bi-endian-support/. Most
    like this code has no interest to anyone except Cisco.

systemtap-old_compiler-task_finder.patch - Older gcc compiler (like gcc 3.4.x)
    produce compilation warning here, incorrectly assuming that dentry may be
    used uninitialized

systemtap-cross_compile_helper.patch - Cross compilation assist. Actually
    include a lot of different small things. Here is the list of them (may not
    be complete):
    x) target system build may use difference 'gcc' and different 'make'.
    Different values could be specified through SYSTEMTAP_TARGET_CC and
    SYSTEMTAP_MAKE environment variable
    x) -y option introduced. Once specified it will indicate that systemtap
    compiler operates in cross compilation mode and value of option serves
    sysroot path value. Stap compiler supports several such options (details
    below)
    x) if compiler in cross compilation mode it does not use PATH and
    LD_LIBRARY_PATH to search for executables and libraries. Instead it uses
    SYSTEMTAP_CROSS_PATH and SYSTEMTAP_CROSS_LD_LIBRARY_PATH environment
    variables. PATH and LD_LIBRARY_PATH are usually different between host and
    target. Note SYSTEMTAP_CROSS_PATH and SYSTEMTAP_CROSS_LD_LIBRARY_PATH could
    be specified as relative of sysroot values, as well as host absolute
    pathes
    x) if stap compiler is in cross compilation mode it does not try to rebuild
    uprobes. It assumes that uprobes.ko is already built and installed in
    proper location in sysroot
    x) find_executable function is replaced with special sysroot class that
    handles different ways how file path needed during stap compilation (some
    times it needs file path in host terms, sometimes it would need it as it
    would be seen on target). Sysroot supports multiple sysroot. For example
    like in openwrt case one may have target root file system and overlay file
    system. Stap compiler in cross compilation mode can search both. Normally
    look up happens as iteration over sysroots and for each sysroot we iterate
    over list of directories specified in SYSTEMTAP_CROSS_PATH and
    SYSTEMTAP_CROSS_LD_LIBRARY_PATH. Quite normal case when one sysroot is
    specified so executables will be searched by adding sysroot value in front
    of directories specified by SYSTEMTAP_CROSS_PATH and
    SYSTEMTAP_CROSS_LD_LIBRARY_PATH.

    It is understood that above description may not be really sufficient to get
    full understanding of this patch. If community is interested I can give
    details explanation and see whether we can rework it. Above I just tried to
    give a taste of what sort of issue we are dealing with when we work in
    cross compiled environment

systemtap-data-in_another_cu.patch - Fix for issue
    6977 http://sourceware.org/bugzilla/show_bug.cgi?id=6977. We found it is
    very annoying and limiting for script writers, so we fixed it. It deals
    with  situation of accessing global variables that are defined in *other*
    compilation units.

systemtap-composed_rootfs_finder.patch - in embedded universe final root file
    system may be composed as link farm to set of files system. Example is
    Openwrt root file system and overlay. Systemtap runtime searches
    executables by canonical name. It won't work for link farm case. For
    link farm case ELF file name compiled into script is used as primary file
    path to check for file in file system or as key in list of file systems
    that participate in composition of final link farm. Outside of Cisco
    patch may have interest to distros like OpenWrt.

systemtap-line_range_issue_fix.patch - memory leak and crash in
    iterate_over_srcfile_lines in case of line ranges.

systemtap-icc_line_number.patch - deal with the way how ICC generates line
    number information.

systemtap-data-in_library.patch - fix issues of accessing global data in
    libraries (opposed to executables) Not sure whether it is applicable to
    latest tree.

systemtap-mips-uprobes.patch - MIPS uprobes kernel module code.

systemtap-mips.patch - MIPS systemtap runtime and and tapset code.

systemtap-ppc32.patch - runtime fixes for powerpc 32bit case.

systemtap-smaller_buffer_size.patch - stap debugfs 1Mb buffer size is too big
    for systems like  WRT54GL, where whole box has 16Mb or 32Mb of memory.
    Change 'b' option handling, so smaller buffers could be specified (i.e in
    Kbytes). Patch tries to be backward compatible with previous option usage
    - if no size letter is specified, it is treated like Megabytes

systemtap-incorrect_kernel_buildid_hack.patch - there are idiosyncrasies in
    Cisco build system when kernel rebuilt twice and buildid in target image
    does not correspond to one we left to deal with on the host. This is hack
    to deal with it. Should be ignored by everyone else.

systemtap-ios_keyword.patch - Cisco specific patch. Should be ignored by
    everybody else. Introduces ios keyword (along with 'process' and 'kernel').
    It deals with the fact that IOS process  canonical name changes depending
    on IOS image feature key. 'ios' keyword allows us to have scripts reusable
    across different target images. Real ios process name is passed with -i
    option by higher level wrapper script.

systemtap-no_userland_prologue_search.patch - Add option '-U' that disables
    function prologue  searches in user-land processes. Current -P affects only
    kernel probes.

systemtap-kernel_source_tree.patch - Option -T allows to specify location of
    kernel sources. Normally needed only in cross compilation environment.

systemtap-uprobes_in_kernel_as_module.patch - Fix for case when in cross
    compilation environment we  may have uprobes built as module in kernel
    build (opposite to built by systemtap). In this case, in order for
    kernel_built_uprobes function to see unregister_uprobe symbol it has to be
    added into kernel_exports even-though it comes from module (not uprobes)

systemtap-remote_hack.patch - allows remote_uris mechanism to work in cross
    compilation environment, where target kernel release won't be the same as
    host kernel release

systemtap-cross_testsuite.patch - Attempt to change testsuite to operate in
    cross compilation environment where access to target happens either through
    dejagnu remote target facilities or through stap --remote mechanism.
    Massive patch, across bunch of test cases, not sure whether community has
    any interest in it. It was used by us to validate systemtap operation on our
    targets. In order to run, it requires stap board specific config (not
    provided here), which would specify things like stap_board_args, and others.

systemtap-enable_vma_tracker.patch - cover corner cases where
    _stp_umodule_relocate is used but vma_tracker is not enabled, while it
    should be, becuase it uses stap_find_vma_map_info_user function.

Thanks,
Victor

On Thu, 23 Aug 2012, Josh Stone wrote:

> On 08/22/2012 01:04 PM, Victor Kamensky wrote:
> > Hi Per, Josh,
> >
> > We (Cisco) have pretty much full systemtap MIPS port.
> [...]
> > We always meant to publish all of it, but never get our hands on it. I
> > hope this thread would be final push :). We will do it. Unfortunately, I
> > cannot publish our patch series right away. I need to get permission from
> > our company legal. It should not be a problem, but it may take two or
> > three weeks.
>
> This is good news, and better that you're spurred now to share it. :)
>
> > Our latest series of patches is based on systemtap-1.6. I'll see whether I
> > can find time and move them to latest git ... In worst case scenario we
> > just publish what we have.
>
> IMO, it would be good to go ahead and post the 1.6 patches directly
> after your legal clears it.  Then if you're willing and able to rebase
> that onto the latest git, that's great, but we don't need to hold up
> everything for that possibility.
>
> Thanks,
> Josh

Attachment: systemtap-1.6-cisco-patches.tar.gz
Description: Binary data