kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #10427
Re: Regression Testing
On 05/07/2013 08:01 AM, Brian Sidebotham wrote:
> On 7 May 2013 13:51, Dick Hollenbeck <dick@xxxxxxxxxxx <mailto:dick@xxxxxxxxxxx>> wrote:
>
> On 05/06/2013 04:36 PM, Brian Sidebotham wrote:
> > This turned into just a set of notes about CTest and Boost.Test:
> >
> > CTest integrates well with CMake [1] and allows us to run executables that perform
> tests.
> > Each test can be set as required (to be passed) or not. The result of each test is based
> > on a rule that is defined in the each tests properties. For example, the integer return
> > code from a process can be tested, or a regex match can be performed on stdout or stderr
> > for the test process.
> >
> > Simple tests that depend on the integer return code of a process can be done via CMake's
> > execute_process by checking RESULT_VARIABLE.
> >
> > Using the process return code, both CTest and CMake can test python API changes by
> > executing python scripts and checking python's return value.
> >
> > The Boost.Test framework provides easy methods for writing C++ test cases as
> executables.
> > A single executable can contain as many tests as required. Each test is called
> > individually as a command line argument. The Boost.Test framework can work by
> returning a
> > different integer return code from the executable depending on the result of a test.
> It is
> > has some additional features which aid automatic regression testing (for example on
> build
> > servers and the like) [2]
> >
> > Writing test cases using the Boost.Test framework provides an easy way to test
> > WorkerFunctions within the codebase as we can write test cases and link them to the
> > classes we're testing. This really requires code to be more modular so that we can link
> > test executables against libraries.
> >
> > CTest and Boost.Test can allow memory leak testing as CTest integrates well with
> Purify or
> > Valgrind.
> >
> > CTest allows subsets of tests to be run, so if you know you've only been working on a
> > certain part of the codebase, you can quickly run just the tests that should be affected
> > by your changes before running the whole test suite.
> >
> > The Python API requires test cases for stability so we know that our changes to the C++
> > code do not break other peoples IP based on our Python API.
>
>
> Thanks very much. I probably will not have much input into this decision moving forward.
>
> How is the wxPython build script coming for Windows?
>
>
> Hi Dick,
>
> I am going to cmake-ify wxPython to build it (I will maintain a patch, and hope wxPython
> will accept it), as I want to get rid of distutils which is really very broken for MinGW.
> Building wxPython is actually relatively straight forward, but there are a lot of options
> which translate to wxWidgets build options which need covering in cmake. This work isn't
> yet off the ground.
>
> I'm currently working on the NSIS packaging of python-a-mingw-us. At the moment it looks
> like we'll have to use an 8-bit colour BMP for the logo which doesn't look great. There
> seems to be some peculiarities with the BMP format used by makensis because if it is
> anything other than an 8-bit BMP the logo doesn't get displayed.
>
> I have read a lot of posts that sort-of relate to this, but most end up building the
> windows installer on a windows machine and this limitation doesn't then appear to exist.
>
> Anyway, I have the installer pretty much up and running. I'd like to email you a diff to
> review before committing, just to make sure you're happy with what I've done. I'll email a
> patch tonight. I had to rename one of the installable components because the name is
> directly used as a variable name in the nsis script, and a few other things too in order
> to get the packaging to work.
Its been weeks now. I did not receive it. I don't see it checked in either.
>
> I think everything except the headers component is a required installation component.
The headers are required for folks to stack C/C++ code on top, such as wxPython. So
unless we want to do a second "development" package, I don't think there's harm in leaving
them in a single package.
Thanks,
Dick
>
> Best Regards, Brian.
Follow ups
References
-
Re: Regression Testing
From: Edwin van den Oetelaar, 2013-04-29
-
Re: Regression Testing
From: Dick Hollenbeck, 2013-04-30
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-04-30
-
Re: Regression Testing
From: Edwin van den Oetelaar, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Miguel Angel Ajo, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Miguel Angel Ajo, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Miguel Angel Ajo, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Brian Sidebotham, 2013-05-06
-
Re: Regression Testing
From: Dick Hollenbeck, 2013-05-07
-
Re: Regression Testing
From: Brian Sidebotham, 2013-05-07