← Back to team overview

kicad-developers team mailing list archive

Re: Regression Testing

 

On 05/07/2013 08:01 AM, Brian Sidebotham wrote:
> On 7 May 2013 13:51, Dick Hollenbeck <dick@xxxxxxxxxxx <mailto:dick@xxxxxxxxxxx>> wrote:
> 
>     On 05/06/2013 04:36 PM, Brian Sidebotham wrote:
>     > This turned into just a set of notes about CTest and Boost.Test:
>     >
>     > CTest integrates well with CMake [1] and allows us to run executables that perform
>     tests.
>     > Each test can be set as required (to be passed) or not. The result of each test is based
>     > on a rule that is defined in the each tests properties. For example, the integer return
>     > code from a process can be tested, or a regex match can be performed on stdout or stderr
>     > for the test process.
>     >
>     > Simple tests that depend on the integer return code of a process can be done via CMake's
>     > execute_process by checking RESULT_VARIABLE.
>     >
>     > Using the process return code, both CTest and CMake can test python API changes by
>     > executing python scripts and checking python's return value.
>     >
>     > The Boost.Test framework provides easy methods for writing C++ test cases as
>     executables.
>     > A single executable can contain as many tests as required. Each test is called
>     > individually as a command line argument. The Boost.Test framework can work by
>     returning a
>     > different integer return code from the executable depending on the result of a test.
>     It is
>     > has some additional features which aid automatic regression testing (for example on
>     build
>     > servers and the like) [2]
>     >
>     > Writing test cases using the Boost.Test framework provides an easy way to test
>     > WorkerFunctions within the codebase as we can write test cases and link them to the
>     > classes we're testing. This really requires code to be more modular so that we can link
>     > test executables against libraries.
>     >
>     > CTest and Boost.Test can allow memory leak testing as CTest integrates well with
>     Purify or
>     > Valgrind.
>     >
>     > CTest allows subsets of tests to be run, so if you know you've only been working on a
>     > certain part of the codebase, you can quickly run just the tests that should be affected
>     > by your changes before running the whole test suite.
>     >
>     > The Python API requires test cases for stability so we know that our changes to the C++
>     > code do not break other peoples IP based on our Python API.
> 
> 
>     Thanks very much.  I probably will not have much input into this decision moving forward.
> 
>     How is the wxPython build script coming for Windows?
> 
> 
> Hi Dick,
> 
> I am going to cmake-ify wxPython to build it (I will maintain a patch, and hope wxPython
> will accept it), as I want to get rid of distutils which is really very broken for MinGW.
> Building wxPython is actually relatively straight forward, but there are a lot of options
> which translate to wxWidgets build options which need covering in cmake. This work isn't
> yet off the ground.
> 
> I'm currently working on the NSIS packaging of python-a-mingw-us. At the moment it looks
> like we'll have to use an 8-bit colour BMP for the logo which doesn't look great. There
> seems to be some peculiarities with the BMP format used by makensis because if it is
> anything other than an 8-bit BMP the logo doesn't get displayed.
> 
> I have read a lot of posts that sort-of relate to this, but most end up building the
> windows installer on a windows machine and this limitation doesn't then appear to exist.
> 
> Anyway, I have the installer pretty much up and running. I'd like to email you a diff to
> review before committing, just to make sure you're happy with what I've done. I'll email a
> patch tonight. I had to rename one of the installable components because the name is
> directly used as a variable name in the nsis script, and a few other things too in order
> to get the packaging to work.

Its been weeks now.  I did not receive it.  I don't see it checked in either.

> 
> I think everything except the headers component is a required installation component.

The headers are required for folks to stack C/C++ code on top, such as wxPython.  So
unless we want to do a second "development" package, I don't think there's harm in leaving
them in a single package.

Thanks,

Dick


> 
> Best Regards, Brian.



Follow ups

References