kicad-developers team mailing list archive

[Brian Sidebotham <brian.sidebotham@xxxxxxxxx>]

On 29 April 2013 01:15, Dick Hollenbeck <dick@xxxxxxxxxxx> wrote:

>
> On Apr 28, 2013 10:54 AM, "Brian Sidebotham" <brian.sidebotham@xxxxxxxxx>
> wrote:
> >
> > Hi Guys,
> >
> > I'm just catching up with the list, and I saw something that caught my
> eye as it's something that's been on my mind for a while:
> >
> > --------------------------
> >
> > Dick Hollenbeck wrote:
> >
> > - Right now, I am finding too many bugs in the software ...
> >
> > - We might do well as a team by slowing down and focusing
> > - on reliability and quality not features for awhile.  Firstly,
> > - the bugs are damaging to the project.
> >
> > ---------------------------
> >
> > I agree with this, there are things I'd like to add to KiCad, but only
> on-top of something I can be confident I'm not breaking, especially by
> creating corner case issues.
> >
> > I would like us to think about regression testing using something like
> CTest (Which would make sense as we're currently using CMake anyway!). We
> could then publish dashboard regression testing results.
> >
> > I'm aware work is going into making eeschema and PCBNEW essentially into
> DLL's, so perhaps it's best to wait until that work is complete before
> starting down this road?
> >
> > In particular I'd like to see regression testing on the DRC, Gerber
> generation, and the Python exposed API. Probably in that order of priority
> too. Certainly the Python API changes are already tripping us up, but only
> when they have already been broken in committed code.
> >
> > Being able to regression test changes to optimisations and code tidying
> will help that move along anyway as you can be more confident in your
> changes having complete coverage once the number of tests increases.
> >
> > I am prepared to say that I'll undertake this work too. Obviously it
> can't start straight away as I'm currently doing work on the Windows
> scripting build system and python-a-mingw-us packaging.
> >
> > Is anyone against regression testing, or have alternatives that would
> achieve similar confidence in committed code? My vote is for regression
> testing.
> >
>
> I fully support the idea.  It will expand the size of the source tree
> significantly over time, and increase maintainence, but these costs are
> dwarfed by the benefits.
>
> Pyhon itself has quite a developed test harness environment with lots of
> tests.  They were helpful in getting a-ming-us up to a certain confidence
> level.  I did not need an understanding of the test harness environment to
> use it.
>
> The other thing I learned was that python can call arbitrary C functions
> in an arbitrary dll, even if they are not swigged.
>
>
Excellent, it's good to see there is support for testing in the codebase. I
realise that this will add significantly to the codebase, but it will mean
that bugs in the core at least can be fixed with good confidence in the
changes not affecting other functionality.

Thanks for your feedback Dick.

Best Regards, Brian.