launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

I've put up a couple more branches with parallel testing support work;
you can see here - https://hudson.wedontsleep.org/job/parallel-test/.
Note that the test counts are currently bogus - the subunit stream is
being damaged, but it looks like all the tests are bing attempted. So
its nearly 1/2 the duration of buildbot, even though we're running on
4 cores: CPU isn't the only bottleneck. We can tune further once we
have things reliable. Even without tuning, if we can halve buildbot
build times, that will be a great help to our dev cycle time.

Remaining things to do:
 - the readonly flag tests
 - librarian
 - something is dropping launchpad_ftest_template
 - memcache
 - appserver http ports (wallyworld reports that this has /100's/ of
tests to update

What can you do?

* Try the branch out and report bugs when things break. You need about
300MB RAM per core in your machine, to run a parallel test run.
  - please report the bugs with a paralleltest flag -
https://bugs.edge.launchpad.net/launchpad-foundations/+bugs?field.tag=paralleltest

* Update something that needs doing from the list above. This should
be, individually, pretty easy:
To make something parallel testing safe:
 - if its a layer based thing, during setUp, generate a unique
port/directory/whatever and use BaseLayer.config.add_section() to
override the config for it. Do the same for appserver_config, if that
is being used.
 - otherwise use a unique resource in the test (e.g. tempfile.*)

So it should be pretty simple now, to churn through and fix stuff.

I still really want to get to what i sketched out as a consistent
interface for transient processes, but for now its ok as long as we
come up with /some/ answer for ephemeral ports (which is probably use
add_section only after the server starts. Or something).

-Rob