launchpad-dev team mailing list archive

[Gary Poster <gary.poster@xxxxxxxxxxxxx>]

Pretty version:
https://docs.google.com/a/canonical.com/document/d/1RNYn2XLJ9FTC7kmM0-5t_XyAzRSj2-ne_T-RqemP0bo/edit#


Parallel test project

Status update for June 13, 2012

Overview

In our last report, we had regressed to no passing (green) test runs.
We are back to reasonably good statistics for fully green test
runs--around 55%--even while we have increased our test stress from 8 up
to 32 threads (in preparation for the 24 thread machines being
purchased).  That’s much closer to our goal of 90% than 0%, where we
were in our last update, but still pretty far away.

Interestingly and positively, we are starting to run out of big bugs to
fix.  We believe we have the fix for bug 992184.  Bug 996729 has been
one of the last big bugs hanging over us, and we are landing a fix for
that today.  Other new bugs have been coming up as we run tests on
32-thread machines, but they have generally been very simple to fix, and
we usually fix them within 24 hours of being reported.

The only other big bug remaining is LXC bug 994752.  Even setting a
timeout to ten minutes is sometimes insufficient for getting 32 LXC
instances started on the 32-thread machine.  I think the answer will
need to include reducing the number of threads we actually start, using
something like testrepository bug 957145.

Purchasing the two new machines for the data center continues forward,
as far as we can tell.

Our slack time work on lpsetup has moved past LEP work
(https://dev.launchpad.net/LEP/LaunchpadSetupScripts) into a confidence
that we can proceed, as well as a recognition that this has evolved from
slack project for a developer (Francesco) into a stretch goal for the
entire squad.  Congratulations to Francesco for moving things forward
enough to get this acceptance and recognition for the project.

Gary is working on LEPs for turning lpsetup into a tool for a Juju
Launchpad developer charm, and for turning that charm into a tool for a
much simpler “ec2 test” command.  We have a bit more proof of concept
work to do, and then will send out a note to the Launchpad dev list.

Benji continues his work on termbeamer, a GTK terminal broadcaster over
a Jabber channel such as GTalk
(https://dev.launchpad.net/yellow/Termbeamer).  Francesco has a Sublime
Text 2 plugin for quickly sending snippets from the editor to the Ubuntu
pastebin.


Progress towards biweekly action items

[yellow] Bug fixes for test failures discovered in parallel test runs:
682771: in progress (gmb)
682772: fix landing (bac)
992184: fixed (gary, benji, stub)
996720: fixed (frankban)
996729: fixed (benji, gary, gmb, frankban).
1001520: fixed (frankban)
1002820: fixed (frankban, benji, bac)
1003040: fixed (frankban)
1007576: fixed (gary)
1007579: fixed (frankban)
1009712: fixed (gmb)
1010251: fixed (frankban)
1011793: fixed (benji)
1011847: fixed (frankban)
1012166: fixed (frankban)
1012314: fixed (frankban)
bin/test --load-list no longer honors randomization (--shuffle): fixed
(frankban)
[yellow] Continued run record at
https://dev.launchpad.net/ParallelTests/ResultsLog. DONE.
[lifeless, flacoste, et al] boxes purchased and configured in data
center.  In progress.


Other accomplishments

(see summary)


Progress on tracked items

Completed by others

987938 (subunit packaging): fixed (jelmer)

New and incomplete

(none)

Carried over and incomplete

944386 (kernel): We have a workaround but IS needs a fix.  apw said he
would look into it.  flacoste escalated.
985812 (Juju)
959352 (LXC): the workaround is doing better than I had feared, so this
is less essential, but I am still afraid this might bite us, so I’m
leaving it around for a little bit longer.


Goals for next meeting

[yellow] Bug fixes for test failures discovered in parallel test runs,
particularly those mentioned as in progress above.
[yellow] Continued run record at
https://dev.launchpad.net/ParallelTests/ResultsLog.
[lifeless, flacoste, et al] boxes purchased and configured in data center.