kicad-developers team mailing list archive

[Wayne Stambaugh <stambaughw@xxxxxxxxxxx>]

On 4/29/2013 8:45 AM, Wayne Stambaugh wrote:
> On 4/28/2013 8:15 PM, Dick Hollenbeck wrote:
>>
>> On Apr 28, 2013 10:54 AM, "Brian Sidebotham" <brian.sidebotham@xxxxxxxxx
>> <mailto:brian.sidebotham@xxxxxxxxx>> wrote:
>>>
>>> Hi Guys,
>>>
>>> I'm just catching up with the list, and I saw something that caught my
>> eye as it's something that's been on my mind for a while:
>>>
>>> --------------------------
>>>
>>> Dick Hollenbeck wrote:
>>>
>>> - Right now, I am finding too many bugs in the software ...
>>>
>>> - We might do well as a team by slowing down and focusing
>>> - on reliability and quality not features for awhile.  Firstly,
>>> - the bugs are damaging to the project.
>>>
>>> ---------------------------
>>>
>>> I agree with this, there are things I'd like to add to KiCad, but only
>> on-top of something I can be confident I'm not breaking, especially by
>> creating corner case issues.
>>>
>>> I would like us to think about regression testing using something like
>> CTest (Which would make sense as we're currently using CMake anyway!).
>> We could then publish dashboard regression testing results.
>>>
>>> I'm aware work is going into making eeschema and PCBNEW essentially
>> into DLL's, so perhaps it's best to wait until that work is complete
>> before starting down this road?
>>>
>>> In particular I'd like to see regression testing on the DRC, Gerber
>> generation, and the Python exposed API. Probably in that order of
>> priority too. Certainly the Python API changes are already tripping us
>> up, but only when they have already been broken in committed code.
>>>
>>> Being able to regression test changes to optimisations and code
>> tidying will help that move along anyway as you can be more confident in
>> your changes having complete coverage once the number of tests increases.
>>>
>>> I am prepared to say that I'll undertake this work too. Obviously it
>> can't start straight away as I'm currently doing work on the Windows
>> scripting build system and python-a-mingw-us packaging.
>>>
>>> Is anyone against regression testing, or have alternatives that would
>> achieve similar confidence in committed code? My vote is for regression
>> testing.
> 
> I think it's good idea as long as we think it through before we start
> implementing them.  I want avoid a free for all mentality and then have
> to go back and clean up the mess.  We should set down some preliminary
> guidelines for testing along the lines of the coding policy before we
> start actually writing test code.  This way developers will no what is
                                                           ^^know^^

Sorry, the caffeine hasn't kicked in yet.

> expected.
> 
>>>
>>
>> I fully support the idea.  It will expand the size of the source tree
>> significantly over time, and increase maintainence, but these costs are
>> dwarfed by the benefits.
>>
>> Pyhon itself has quite a developed test harness environment with lots of
>> tests.  They were helpful in getting a-ming-us up to a certain
>> confidence level.  I did not need an understanding of the test harness
>> environment to use it.
>>
>> The other thing I learned was that python can call arbitrary C functions
>> in an arbitrary dll, even if they are not swigged.
> 
> I've used the Python unit test framework for testing Python code.  I
> never thought about using it for testing C++ code but it does seem
> feasible.  Does anybody have any experience doing this or are there any
> examples of anyone else doing this so we can get an idea of what is
> involved?  If the Python testing framework is not adequate, we can
> always take a look at the Boost testing framework.  I looked at this
> several months ago and it looked like a good fit given that we already
> have Boost as a dependency and it's feature parity seemed as good as any
> of the other open source C/C++ testing frameworks and even many of the
> commercial testing frameworks.
> 
>>
>>> We would need good support from all, new code requires new tests, and
>> nobody is better suited to devising the tests than the person who
>> specified the new functionality - which means all developers would
>> probably end up writing test cases at some point.
>>>
>>> Best Regards, Brian.
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~kicad-developers
>>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> <mailto:kicad-developers@xxxxxxxxxxxxxxxxxxx>
>>> Unsubscribe : https://launchpad.net/~kicad-developers
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~kicad-developers
>> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~kicad-developers
>> More help   : https://help.launchpad.net/ListHelp
>>
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~kicad-developers
> Post to     : kicad-developers@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~kicad-developers
> More help   : https://help.launchpad.net/ListHelp
>