← Back to team overview

kicad-developers team mailing list archive

Re: Regression Testing

 

On 7 May 2013 13:51, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:

> On 05/06/2013 04:36 PM, Brian Sidebotham wrote:
> > This turned into just a set of notes about CTest and Boost.Test:
> >
> > CTest integrates well with CMake [1] and allows us to run executables
> that perform tests.
> > Each test can be set as required (to be passed) or not. The result of
> each test is based
> > on a rule that is defined in the each tests properties. For example, the
> integer return
> > code from a process can be tested, or a regex match can be performed on
> stdout or stderr
> > for the test process.
> >
> > Simple tests that depend on the integer return code of a process can be
> done via CMake's
> > execute_process by checking RESULT_VARIABLE.
> >
> > Using the process return code, both CTest and CMake can test python API
> changes by
> > executing python scripts and checking python's return value.
> >
> > The Boost.Test framework provides easy methods for writing C++ test
> cases as executables.
> > A single executable can contain as many tests as required. Each test is
> called
> > individually as a command line argument. The Boost.Test framework can
> work by returning a
> > different integer return code from the executable depending on the
> result of a test. It is
> > has some additional features which aid automatic regression testing (for
> example on build
> > servers and the like) [2]
> >
> > Writing test cases using the Boost.Test framework provides an easy way
> to test
> > WorkerFunctions within the codebase as we can write test cases and link
> them to the
> > classes we're testing. This really requires code to be more modular so
> that we can link
> > test executables against libraries.
> >
> > CTest and Boost.Test can allow memory leak testing as CTest integrates
> well with Purify or
> > Valgrind.
> >
> > CTest allows subsets of tests to be run, so if you know you've only been
> working on a
> > certain part of the codebase, you can quickly run just the tests that
> should be affected
> > by your changes before running the whole test suite.
> >
> > The Python API requires test cases for stability so we know that our
> changes to the C++
> > code do not break other peoples IP based on our Python API.
>
>
> Thanks very much.  I probably will not have much input into this decision
> moving forward.
>
> How is the wxPython build script coming for Windows?


Hi Dick,

I am going to cmake-ify wxPython to build it (I will maintain a patch, and
hope wxPython will accept it), as I want to get rid of distutils which is
really very broken for MinGW. Building wxPython is actually relatively
straight forward, but there are a lot of options which translate to
wxWidgets build options which need covering in cmake. This work isn't yet
off the ground.

I'm currently working on the NSIS packaging of python-a-mingw-us. At the
moment it looks like we'll have to use an 8-bit colour BMP for the logo
which doesn't look great. There seems to be some peculiarities with the BMP
format used by makensis because if it is anything other than an 8-bit BMP
the logo doesn't get displayed.

I have read a lot of posts that sort-of relate to this, but most end up
building the windows installer on a windows machine and this limitation
doesn't then appear to exist.

Anyway, I have the installer pretty much up and running. I'd like to email
you a diff to review before committing, just to make sure you're happy with
what I've done. I'll email a patch tonight. I had to rename one of the
installable components because the name is directly used as a variable name
in the nsis script, and a few other things too in order to get the
packaging to work.

I think everything except the headers component is a required installation
component.

Best Regards, Brian.

Follow ups

References