kicad-developers team mailing list archive

[Brian Sidebotham <brian.sidebotham@xxxxxxxxx>]

On 17 July 2013 02:00, Kaspar Emanuel <kaspar.emanuel@xxxxxxxxx> wrote:

> Hi all,
>
> I had a brief look at using Python for unit testing today and wrote a few
> tests for the EDA_RECT class.
>
> I added a tests directory in the root-dir and in it there is a README.md
> that tells you how to run the tests. There should be no dependencies except
> Python 2.7 . If I go further with this I would likely switch to the
> nose-test module.
>
> Any feedback welcome. What I really need is some help in a plan to asses
> the usefulness and feasibility of this. Can you see yourself using this?
> Where could this Python approach falter?
>
>  Here is my branch:
> https://code.launchpad.net/~kaspar-emanuel/kicad/python-unittest
>
> Ciao,
>
> Kaspar
>

Hi Kaspar,

Thanks for doing some initial work on the regression testing in KiCad. I am
still stuck with doing work on python scripting support for KiCad on
Windows, but I should be at a place soon where that work can be released.
In which case, I can turn my attention to creating a branch for working out
test strategies.

This initial python work looks good. One of the things that instigated
regression testing was to include the python API so that we could be sure
when changing things in the C++ code we weren't inadvertently breaking the
Python API and therefore breaking lots of scripts. So this work is always
going to be needed.

We also need to be able to test the UI code too. We need a way of testing
various plotting options and settings are always honoured so that we don't
break the essentials of KiCad such as plotting.

"Using" whatever framework we come up with should ideally become
non-optional to developers. Therefore the testing framework should be:

(a) QUICK to use. This doesn't necessarily mean that running all tests in
the entire test suite takes a short amount of time, but rather all tests
are performed as quickly as possible whilst also allowing an easy selection
of subsets of tests to be run. Developers can always run all tests, but can
save time by running tests only relevant to what they've been working on.

(b) OBVIOUS assertions. Assertions should make it obvious what failed.
There should be good information pointing to the source of the problem*

(c) MINIMAL overhead. Writing new tests should be very quick. Once you
start writing tests you realise that most functions require several tests.
We might even want to have some template of tests that need to be performed
on certain return types or input types, etc.

* This can mean the order of tests will need thinking about. I would rather
have an assertion on a WorkerFunction as the first fatal error report as
opposed to a more complicated UI assertion which uses the failing
WorkerFunction underneath.

Best Regards, Brian.