kicad-developers team mailing list archive

[Brian Sidebotham <brian.sidebotham@xxxxxxxxx>]

This turned into just a set of notes about CTest and Boost.Test:

CTest integrates well with CMake [1] and allows us to run executables that
perform tests. Each test can be set as required (to be passed) or not. The
result of each test is based on a rule that is defined in the each tests
properties. For example, the integer return code from a process can be
tested, or a regex match can be performed on stdout or stderr for the test
process.

Simple tests that depend on the integer return code of a process can be
done via CMake's execute_process by checking RESULT_VARIABLE.

Using the process return code, both CTest and CMake can test python API
changes by executing python scripts and checking python's return value.

The Boost.Test framework provides easy methods for writing C++ test cases
as executables. A single executable can contain as many tests as required.
Each test is called individually as a command line argument. The Boost.Test
framework can work by returning a different integer return code from the
executable depending on the result of a test. It is has some additional
features which aid automatic regression testing (for example on build
servers and the like) [2]

Writing test cases using the Boost.Test framework provides an easy way to
test WorkerFunctions within the codebase as we can write test cases and
link them to the classes we're testing. This really requires code to be
more modular so that we can link test executables against libraries.

CTest and Boost.Test can allow memory leak testing as CTest integrates well
with Purify or Valgrind.

CTest allows subsets of tests to be run, so if you know you've only been
working on a certain part of the codebase, you can quickly run just the
tests that should be affected by your changes before running the whole test
suite.

The Python API requires test cases for stability so we know that our
changes to the C++ code do not break other peoples IP based on our Python
API.

[1] http://www.cmake.org/cmake/help/cmake_tutorial.html
[2]
http://www.boost.org/doc/libs/1_53_0/libs/test/doc/html/utf/usage-recommendations/command-line-specific.html

Best Regards, Brian.