launchpad-dev team mailing list archive

[Gary Poster <gary.poster@xxxxxxxxxxxxx>]

Also readable in google docs:
https://docs.google.com/document/d/12zW9D79nxHrY0KCjfETpS6RLrjcOo95MygenBfClX04/edit

Parallel test project

Status update for May 30, 2012

Overview

As warned by the Yellow retrospective meeting minutes from this past
Friday, this was a tough couple of weeks for the project.  An easy to
understand statistic that shows our difficulties is that the last green
test run we had was Monday, May 21, nine days ago.

In particular, three problems wrecked the schedule and continue to have
some echoes even now.  Bugs 1004088, 996729 (with fallout friends
1003206 and 1003696), and 1004625 were primarily responsible for our
troubles.  We’re now down to eight or nine failures, all newly revealed
thanks to bug 1004625.

Therefore, we did not get anywhere near a 90% green test run, which we
had hoped for.  We simply didn’t run tests for a big chunk of the period
because we knew things were broken.

After acknowledging all of these difficulties, it is worth noting that
we did meet all of the specific goals from the last report.  A highlight
there is that Robert, Francis, and IS determined what machines we would
purchase, and that is in progress.  Additionally, we closed more test
failure bugs.

Finally, on the slack time front, Francesco created a LEP for lpsetup
and gathered feedback about it from the Launchpad team and Robert
(https://dev.launchpad.net/LEP/LaunchpadSetupScripts); Graham worked
with Laura on a blog post about the Launchpad bug fixing outreach they
did at UDS (blog is not yet released); and Benji released a beta package
of termbeamer, a GTK terminal broadcaster over a Jabber channel such as
GTalk (https://dev.launchpad.net/yellow/Termbeamer).
Progress towards biweekly action items

[yellow] Bug fixes for test failures discovered in parallel test runs:
992814: could not replicate.  Marked incomplete for now.
993510: fixed (frankban)
996720: NO PROGRESS, but should be a simple timing issue
996729: fix failed, causing 1003206 and 1998003696.  IN PROGRESS (gmb,
frankban)
998040: fixed (benji)
999770: dupe of 996729
1002796: fixed (frankban)
1002820: failed fix so IN PROGRESS (benji, bac)
1003044: fixed (benji)
1003097: fixed (frankban)
1003200: fixed (bac)
1004088: fixed (bac, gmb)
1004625: fixed (bac)
Unicode decode error in our buildbot/subunit integration code: done
(frankban)
Cleaning out old testr runs in our buildbot/testr integration code: (bac)
[yellow] Test results of two slaves on same machine, with and without
CPU contention: DONE for no contention, attempted for contention, and
rejected.
[yellow] Continued run record at
https://dev.launchpad.net/ParallelTests/ResultsLog. DONE, to the degree
that we were able to run them
[lifeless, flacoste, et al] Determine boxes desired in data center for
tests and arrange for purchase and set up.  DONE.
Other accomplishments

[Francesco] lpsetup LEP, work on lpsetup
[Graham] UDS blog
[Benji] termbeamer (see summary)
Progress on tracked items

Completed by others

RT 50242 - get a buildbot machine for testing (gnuoy and gary_poster)
New and incomplete

998040/593536 (LP/bzr): we need to upgrade LP to a newer bzr.  This
reportedly has difficulties.  Blue squad is investigating.
Carried over and incomplete

944386 (kernel): We have a workaround but IS needs a fix.  apw said he
would look into it.  flacoste escalated.
987938 (subunit packaging): jelmer reports that he did it; we need to
verify.
985812 (Juju)
959352 (LXC): the workaround is doing better than I had feared, so this
is less essential, but I am still afraid this might bite us, so I’m
leaving it around for a little bit longer.

Goals for next meeting

[yellow] Bug fixes for test failures discovered in parallel test runs,
particularly those mentioned as in progress above.
[yellow] Continued run record at
https://dev.launchpad.net/ParallelTests/ResultsLog.
[lifeless, flacoste, et al] boxes purchased and configured in data center.