Test Suites

To Contents

To previous page

To next page

 




Test Suites

The eCos kernel and other packages have test suites that rigorously exercise the available features and confirm correct execution. The tests are run on many different possible configurations, but the high number of configuration permutations makes it impossible to test them all. The use of test suites is particularly important for embedded systems, where software robustness is a priority. All eCos software is tested prior to shipping, but if you define your own configuration, you will probably want to verify that the test cases work for it.

This release includes test suites for the eCos kernel, kernel C API, C library, ITRON compatibility, and device driver packages. The use of the test suites is similar for all packages. The tests are supplied as source code for building with your specific eCos configurations. The test case source code is located under the base source directory BASE_DIR/packages/:

There may be additional tests found in other packages.

Each test suite consists of a number of test cases which can be executed individually. A test case may involve one or more individual tests of the package's features. Successful completion of each test within the test case is reported as a line of text that is sent to the diagnostic channel (usually the serial port) for display on a terminal or terminal emulator.

Each test case runs only once and usually requires target hardware to be reset on completion. Note that certain test cases may not terminate immediately, especially if they involve delays and run on a target simulator.

Using the Configuration Tool

Using the eCos Configuration Tool it is possible to automate the downloading and execution of tests with the appropriately configured eCos packages. To do so, compile and link the test cases by using the Build->Tests menu item, after which the tests can be downloaded and executed by selecting Tools->Run Tests .

When a test run is invoked, a resizable property sheet is displayed, comprising three tabs: Executables , Output and Summary .

Three buttons appear on the property sheet itself: Run/Stop , Close and Properties .

The Run button is used to initiate a test run. Those tests selected on the Executables tab are run, and the output recorded on the Output and Summary tabs. During the course of a run, the Run button changes to Stop . This button may be used to interrupt a test run at any point.

See the eCos User's Guide for further details.

Using the command line

At the moment, there is no tool for automating testing on Linux, so you will have to run the tests manually.

It may also be necessary to run tests by hand if the automated tool finds any failing tests: it may be necessary to diagnose the problem by debugging the test.

Build the tests by typing ™make tests' in the root of the build directory. This will cause the tests to be built and installed under <install-path>/tests/ .

Running the test manually is done simply by invoking GDB, connecting to the target, downloading the test, optionally setting some breakpoints, and then running the test. All this was covered in Target Setup .

Testing Filters

While most test cases today run solely in the target environment, some packages may require external testing infrastructure and/or feedback from the external environment to do complete testing.

The serial package is an example of this. It is the first package to require external testing infrastructure, but it will certainly not be the last.

Since the serial line is also used for communication with GDB, a filter is inserted in the communication pathway between GDB and the serial device which is connected to the hardware target. The filter forwards all communication between the two, but also listens for special commands embedded in the data stream from the target.

When such a command is seen, the filter stops forwarding data to GDB from the target and enters a special mode. In this mode the test case running on the target is able to control the filter, commanding it to run various tests. While these tests run, GDB is isolated from the target.

As the test completes (or if the filter detects a target crash) the communication path between GDB and the hardware target is re-established, allowing GDB to resume control.

In theory, it is possible to extend the filter to provide a generic framework for other target-external testing components, thus decoupling the testing infrastructure from the (possibly limited) communication means provided by the target (serial, JTAG, Ethernet, etc).

Another advantage is that the host tools will not need to know about the various testing environments required by the eCos packages, since all contact with the target will continue to happen via GDB.

It remains to be seen if it will be possible, or sensible, to implement all target-external testing infrastructure via filters.


Test Suites

To Contents

To previous page

To next page