kicad-developers team mailing list archive
-
kicad-developers team
-
Mailing list archive
-
Message #10381
Re: Regression Testing
On 5/7/2013 9:01 AM, Brian Sidebotham wrote:
> On 7 May 2013 13:51, Dick Hollenbeck <dick@xxxxxxxxxxx
> <mailto:dick@xxxxxxxxxxx>> wrote:
>
> On 05/06/2013 04:36 PM, Brian Sidebotham wrote:
> > This turned into just a set of notes about CTest and Boost.Test:
> >
> > CTest integrates well with CMake [1] and allows us to run
> executables that perform tests.
> > Each test can be set as required (to be passed) or not. The result
> of each test is based
> > on a rule that is defined in the each tests properties. For
> example, the integer return
> > code from a process can be tested, or a regex match can be
> performed on stdout or stderr
> > for the test process.
> >
> > Simple tests that depend on the integer return code of a process
> can be done via CMake's
> > execute_process by checking RESULT_VARIABLE.
> >
> > Using the process return code, both CTest and CMake can test
> python API changes by
> > executing python scripts and checking python's return value.
> >
> > The Boost.Test framework provides easy methods for writing C++
> test cases as executables.
> > A single executable can contain as many tests as required. Each
> test is called
> > individually as a command line argument. The Boost.Test framework
> can work by returning a
> > different integer return code from the executable depending on the
> result of a test. It is
> > has some additional features which aid automatic regression
> testing (for example on build
> > servers and the like) [2]
> >
> > Writing test cases using the Boost.Test framework provides an easy
> way to test
> > WorkerFunctions within the codebase as we can write test cases and
> link them to the
> > classes we're testing. This really requires code to be more
> modular so that we can link
> > test executables against libraries.
> >
> > CTest and Boost.Test can allow memory leak testing as CTest
> integrates well with Purify or
> > Valgrind.
> >
> > CTest allows subsets of tests to be run, so if you know you've
> only been working on a
> > certain part of the codebase, you can quickly run just the tests
> that should be affected
> > by your changes before running the whole test suite.
> >
> > The Python API requires test cases for stability so we know that
> our changes to the C++
> > code do not break other peoples IP based on our Python API.
You pretty much came to the same conclusion I did when I looked at this.
Use CTest to build, run, and manage the tests and use the Boost unit
test framework to write the tests. It might be a good idea to set up a
separate branch on Launchpad until we have a workable testing framework
in place and then merge that into the testing branch. I also want to
make sure we have a reasonable set of expectations that we can either
add to the coding policy or create a separate test policy before we make
it available for general consumption.
>
>
> Thanks very much. I probably will not have much input into this
> decision moving forward.
>
> How is the wxPython build script coming for Windows?
>
>
> Hi Dick,
>
> I am going to cmake-ify wxPython to build it (I will maintain a patch,
> and hope wxPython will accept it), as I want to get rid of distutils
> which is really very broken for MinGW. Building wxPython is actually
> relatively straight forward, but there are a lot of options which
> translate to wxWidgets build options which need covering in cmake. This
> work isn't yet off the ground.
I apologize for not being able to help with CMake-ifying wxPython. I've
been busy and the footprint library table code is proving to be way more
work than I had planned. Thank you for taking the lead on this.
Best Wishes
Wayne
>
> I'm currently working on the NSIS packaging of python-a-mingw-us. At the
> moment it looks like we'll have to use an 8-bit colour BMP for the logo
> which doesn't look great. There seems to be some peculiarities with the
> BMP format used by makensis because if it is anything other than an
> 8-bit BMP the logo doesn't get displayed.
>
> I have read a lot of posts that sort-of relate to this, but most end up
> building the windows installer on a windows machine and this limitation
> doesn't then appear to exist.
>
> Anyway, I have the installer pretty much up and running. I'd like to
> email you a diff to review before committing, just to make sure you're
> happy with what I've done. I'll email a patch tonight. I had to rename
> one of the installable components because the name is directly used as a
> variable name in the nsis script, and a few other things too in order to
> get the packaging to work.
>
> I think everything except the headers component is a required
> installation component.
>
> Best Regards, Brian.
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help : https://help.launchpad.net/ListHelp
>
Follow ups
References
-
Re: Regression Testing
From: Edwin van den Oetelaar, 2013-04-29
-
Re: Regression Testing
From: Dick Hollenbeck, 2013-04-30
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-04-30
-
Re: Regression Testing
From: Edwin van den Oetelaar, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Miguel Angel Ajo, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Miguel Angel Ajo, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Miguel Angel Ajo, 2013-05-01
-
Re: Regression Testing
From: Lorenzo Marcantonio, 2013-05-01
-
Re: Regression Testing
From: Brian Sidebotham, 2013-05-06
-
Re: Regression Testing
From: Dick Hollenbeck, 2013-05-07
-
Re: Regression Testing
From: Brian Sidebotham, 2013-05-07