Debugging Techniques

To Contents

To previous page

To next page

 




Debugging Techniques

eCos applications and components can be debugged in traditional ways, with printing statements and debugger single-stepping, but there are situations in which these techniques cannot be used. One example of this is when a program is getting data at a high rate from a real-time source, and cannot be slowed down or interrupted.

eCos's infrastructure module provides a tracing formalism, allowing the kernel's tracing macros to be configured in many useful ways. eCos's kernel provides instrumentation buffers which also collect specific (configurable) data about the system's history and performance.

Tracing

To use eCos's tracing facilities you must first configure your system to use tracing. You should enable the Asserts and Tracing component ( CYGPKG_INFRA_DEBUG ) and the Use tracing component within it ( CYGDBG_USE_TRACING ). These options can be enabled with the Configuration Tool or by editing the file BUILD_DIR /pkgconf/infra.h manually.

You should then examine all the tracing-related options in the Package: Infrastructure chapter of the eCos Reference Manual. One useful set of configuration options are: CYGDBG_INFRA_DEBUG_FUNCTION_REPORTS and CYGDBG_INFRA_DEBUG_TRACE_MESSAGE, which are both enabled by default when tracing is enabled.

The following "Hello world with tracing" shows the output from running the hello world program (from the programming tutorial in Getting Started with eCos ) that was built with tracing enabled:

Hello world with tracing

 
$ mips-tx39-elf-run --board=jmr3904 hello

Hello, eCos world!
ASSERT FAIL: <2>cyg_trac.h          [ 623] Cyg_TraceFunction_Report_::set_exitvoid()                                                            exitvoid used in typed function
TRACE: <1>mlqueue.cxx         [ 395] Cyg_ThreadQueue_Implementation::enqueue()                                                            {{enter
TRACE: <1>mlqueue.cxx         [ 395] Cyg_ThreadQueue_Implementation::enqueue()                                                            }}RETURNING UNSET!
TRACE: <1>mlqueue.cxx         [ 126] Cyg_Scheduler_Implementation::add_thread()                                                           }}RETURNING UNSET!
TRACE: <1>thread.cxx          [ 654] Cyg_Thread::resume()                                                                                 }}return void
TRACE: <1>cstartup.cxx        [ 160] cyg_iso_c_start()                                                                                    }}return void
TRACE: <1>startup.cxx         [ 142] cyg_package_start()                                                                                  }}return void
TRACE: <1>startup.cxx         [ 150] cyg_user_start()                                                                                     {{enter
TRACE: <1>startup.cxx         [ 150] cyg_user_start()                                                                                     (((void)))
TRACE: <1>startup.cxx         [ 153] cyg_user_start()                                                                                     'This is the system default cyg_user_start()'
TRACE: <1>startup.cxx         [ 157] cyg_user_start()                                                                                     }}return void
TRACE: <1>sched.cxx           [ 212] Cyg_Scheduler::start()                                                                               {{enter
TRACE: <1>mlqueue.cxx         [ 102] Cyg_Scheduler_Implementation::schedule()                                                             {{enter
TRACE: <1>mlqueue.cxx         [ 437] Cyg_ThreadQueue_Implementation::highpri()                                                            {{enter
TRACE: <1>mlqueue.cxx         [ 437] Cyg_ThreadQueue_Implementation::highpri()                                                            }}RETURNING UNSET!
TRACE: <1>mlqueue.cxx         [ 102] Cyg_Scheduler_Implementation::schedule()                                                             }}RETURNING UNSET!
TRACE: <2>intr.cxx            [ 450] Cyg_Interrupt::enable_interrupts()                                                                   {{enter
TRACE: <2>intr.cxx            [ 450] Cyg_Interrupt::enable_interrupts()                                                                   }}RETURNING UNSET!
TRACE: <2>thread.cxx          [  69] Cyg_HardwareThread::thread_entry()                                                                   {{enter
TRACE: <2>cstartup.cxx        [ 127] invoke_main()                                                                                        {{enter
TRACE: <2>cstartup.cxx        [ 127] invoke_main()                                                                                        ((argument is ignored))
TRACE: <2>dummyxxmain.cxx     [  60] __main()                                                                                             {{enter
TRACE: <2>dummyxxmain.cxx     [  60] __main()                                                                                             (((void)))
TRACE: <2>dummyxxmain.cxx     [  63] __main()                                                                                             'This is the system default __main()'
TRACE: <2>dummyxxmain.cxx     [  67] __main()                                                                                             }}return void
TRACE: <2>memcpy.c            [ 112] _memcpy()                                                                                            {{enter
TRACE: <2>memcpy.c            [ 112] _memcpy()                                                                                            ((dst=80002804, src=BFC14E58, n=19))
TRACE: <2>memcpy.c            [ 164] _memcpy()                                                                                            }}returning 80002804
TRACE: <2>cstartup.cxx        [ 137] invoke_main()                                                                                        'main() has returned with code 0. Calling exit()'
TRACE: <2>exit.cxx            [  71] __libc_exit()                                                                                        {{enter
TRACE: <2>exit.cxx            [  71] __libc_exit()                                                                                        ((status=0 ))
TRACE: <2>atexit.cxx          [  84] cyg_libc_invoke_atexit_handlers()                                                                    {{enter
TRACE: <2>atexit.cxx          [  84] cyg_libc_invoke_atexit_handlers()                                                                      (((void)))

Scheduler:

Lock:                0
Current Thread:      <null>

Threads:

Idle Thread          pri =  31 state = R      id =   1
                     stack base = 800021F0 ptr = 80002510 size = 00000400
                     sleep reason NONE     wake reason NONE    
                     queue = 80000C54      wait info = 00000000

<null>               pri =   0 state = R      id =   2
                     stack base = 80002A48 ptr = 8000A968 size = 00008000
                     sleep reason NONE     wake reason NONE    
                     queue = 80000BD8      wait info = 00000000

	  

Kernel Instrumentation

Instrument buffers can be used to find out how many events of a given type happened in the kernel during execution of a program.

You can monitor a class of several types of events, or you can just look at individual events.

Examples of events that can be monitored are:

Examples of fine-grained scheduler event types are:

Information about the events is stored in an event record. The structure that defines this record has type struct Instrument_Record :

The list of records is stored in an array called instrument_buffer , which you can let the kernel provide or you can provide yourself by setting the configuration option CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER.

To write a program that examines the instrumentation buffers:

  1. Enable instrumentation buffers in the eCos kernel configuration. The component macro is CYGPKG_KERNEL_INSTRUMENT.
  2. To allocate the buffers yourself, enable the configuration option CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER.
  3. Include the header file cyg/kernel/instrmnt.h .
    #include <cyg/kernel/instrmnt.h>
  4. The Instrumentation_Record structure is not published in the kernel header file. In the future there will be a cleaner mechanism to access it, but for now you should paste into your code in the following lines:
    struct Instrument_Record
    {
    CYG_WORD16 type; // record type
    CYG_WORD16 thread; // current thread id
    CYG_WORD timestamp; // 32 bit timestamp
    CYG_WORD arg1; // first arg
    CYG_WORD arg2; // second arg
    };
  5. Enable the events you want to record using cyg_instrument_enable() , and disable them later. Look at cyg/kernel/instrmnt.h and the examples below to see what events can be enabled.
  6. Place the code you want to debug between the matching functions cyg_instrument_enable() and cyg_instrument_disable() .
  7. Examine the buffer. For now you need to look at the data in there (the example program below shows how to do that), and future versions of eCos will include a host-side tool to help you understand the data.

Using instrument buffers

This program is also provided in the examples directory.

 
/* this is a program which uses eCos instrumentation buffers; it needs
 to be linked with a kernel which was compiled with support for
 instrumentation */

#include <stdio.h>
#include <pkgconf/kernel.h>
#include <cyg/kernel/instrmnt.h>
#include <cyg/kernel/kapi.h>

#ifndef CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER
# error You must configure eCos with CYGVAR_KERNEL_INSTRUMENT_EXTERNAL_BUFFER
#endif

struct Instrument_Record
{
 CYG_WORD16 type; // record type
 CYG_WORD16 thread; // current thread id
 CYG_WORD timestamp; // 32 bit timestamp
 CYG_WORD arg1; // first arg
 CYG_WORD arg2; // second arg
};

struct Instrument_Record instrument_buffer[20];
cyg_uint32 instrument_buffer_size = 20;

int main(void)
{
 int i;

 cyg_instrument_enable(CYG_INSTRUMENT_CLASS_CLOCK, 0);
 cyg_instrument_enable(CYG_INSTRUMENT_CLASS_THREAD, 0);
 cyg_instrument_enable(CYG_INSTRUMENT_CLASS_ALARM, 0);

 printf("Program to play with instrumentation buffer\n");

 cyg_thread_delay(2);

 cyg_instrument_disable(CYG_INSTRUMENT_CLASS_CLOCK, 0);
 cyg_instrument_disable(CYG_INSTRUMENT_CLASS_THREAD, 0);
 cyg_instrument_disable(CYG_INSTRUMENT_CLASS_ALARM, 0);

 for (i = 0; i < instrument_buffer_size; ++i) {
 printf("Record %02d: type 0x%04x, thread %d, ",
	i, instrument_buffer[i].type, instrument_buffer[i].thread);
 printf("time %5d, arg1 0x%08x, arg2 0x%08x\n",
	instrument_buffer[i].timestamp, instrument_buffer[i].arg1,
	instrument_buffer[i].arg2);
 }
 return 0;
}

Here is how you could compile and run this program in the examples directory, using (for example) the MN10300 compiler:

 
$ make XCC=mn10300-elf-gcc  PKG_INSTALL_DIR
=/tmp/ecos-work-mn10300/install instrument-test
mn10300-elf-gcc -c -o instrument-test.o -g -Wall -I/tmp/ecos-work-mn10300/install/include -ffunction-sections -fdata-sections instrument-test.c
mn10300-elf-gcc -nostartfiles -L/tmp/ecos-work-mn10300/install/lib -W1,--gc-sections -o instrument-test instrument-test.o -Ttarget.ld -nostdlib
$ mn10300-elf-run --board=stdeval1 instrument-test

Instrument buffer output

Here is the output of the instrument-test program. Notice that in little over 2 seconds, and with very little activity, and with few event types enabled, it gathered 17 records. In larger programs it will be necessary to select very few event types for debugging.

 
Program to play with instrumentation buffer
Record 00: type 0x0207, thread 2, time  6057, arg1 0x48001cd8, arg2 0x00000002
Record 01: type 0x0202, thread 2, time  6153, arg1 0x48001cd8, arg2 0x00000000
Record 02: type 0x0904, thread 2, time  6358, arg1 0x48001d24, arg2 0x00000000
Record 03: type 0x0905, thread 2, time  6424, arg1 0x00000002, arg2 0x00000000
Record 04: type 0x0906, thread 2, time  6490, arg1 0x00000000, arg2 0x00000000
Record 05: type 0x0901, thread 2, time  6608, arg1 0x48009d74, arg2 0x48001d24
Record 06: type 0x0201, thread 2, time  6804, arg1 0x48001cd8, arg2 0x480013e0
Record 07: type 0x0803, thread 1, time    94, arg1 0x00000000, arg2 0x00000000
Record 08: type 0x0801, thread 1, time   361, arg1 0x00000000, arg2 0x00000000
Record 09: type 0x0802, thread 1, time   548, arg1 0x00000001, arg2 0x00000000
Record 10: type 0x0803, thread 1, time    94, arg1 0x00000000, arg2 0x00000000
Record 11: type 0x0801, thread 1, time   361, arg1 0x00000001, arg2 0x00000000
Record 12: type 0x0903, thread 1, time   513, arg1 0x48009d74, arg2 0x48001d24
Record 13: type 0x0208, thread 1, time   588, arg1 0x00000000, arg2 0x00000000
Record 14: type 0x0203, thread 1, time   697, arg1 0x48001cd8, arg2 0x480013e0
Record 15: type 0x0802, thread 1, time   946, arg1 0x00000002, arg2 0x00000000
Record 16: type 0x0201, thread 1, time  1083, arg1 0x480013e0, arg2 0x48001cd8
Record 17: type 0x0000, thread 0, time     0, arg1 0x00000000, arg2 0x00000000
Record 18: type 0x0000, thread 0, time     0, arg1 0x00000000, arg2 0x00000000
Record 19: type 0x0000, thread 0, time     0, arg1 0x00000000, arg2 0x00000000

Debugging Techniques

To Contents

To previous page

To next page