coapp-developers team mailing list archive

[Philip Allison <mangobrain@xxxxxxxxxxxxxx>]

I know QA isn't quite the same thing as testing, but would there be
the resources floating around to have something like a continuous
integration server kicking around?  It's a bit buzz-wordy, but knowing
a few minutes after you've pushed something* that you haven't broken
the build can be useful.  More to the point, that kind of set-up is
the first step towards automated regression testing.

Category A is a bunch of command-line tools, whose output should not -
generally speaking - change when given the same input twice (please
correct me if I'm wrong).  It also ought to be fairly easy to script
runs of them with known input.  Each time someone comes up with a new
idea for a unit test, often a simplified version of something which
was observed breaking in the "real world", we come up with a simple
script which runs the relevant tools on the relevant inputs and checks
the output.  The script itself only outputs success or failure.  Run
these scripts each build, as part of the continuous integration
process, and you automatically know on a commit-by-commit basis that
not only does the code build, but it works at least as well as it did
before**.

With any decent DCVS it should be fairly easy for people to avoid
pushing half-finished work without necessarily grinding to a halt.  I
could go on and on about various kinds of workflow and branching
strategies, but that's a topic for another day...

Regards,
Phil

* Sorry, using git terminology here.  Haven't used bazaar, but I use
git /a lot/ both at home and at work.
** Within the bounds of the available tests, obviously.  Not perfect
but better than nothing.