launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

Jono has sent me this to read over before he sent it out to you guys, but 
since it's late and I can't be arsed*, here it is:

(* well, I did read it a little, he might have been calling me names in it)

================
Julian and I had meant to send an email every day. Sadly, our good
intentions dashed on the rocky shores of diligence, but the tugboat of
hope has pulled us out into trade winds of victory, allowing us to run
the colours of reliability up the mainmast of destiny.

If you've followed our previous emails, you'll know we're re-working
the buildmaster, the bit of Launchpad that takes difficult jobs (i.e.
building packages, recipes and translations template) and delegates
them to a farm of machines that are safely sandboxed and regularly
cleaned.

As it stands in trunk, the buildmaster is a bit hard to follow, and
has some reliability and transparency issues. Our goal this week has
been to change it to use Twisted in a normal way.

It's been a big, big, task. Once you change one method to be  * The branch 
needs to be tested big time on dog food.
asynchronous, everything that calls that method has to change. We've
had to look at IBuilder, all of the build farm behavior jobs and many
tests, as well as the buildmaster itself. However, we're pretty much
done.

One thing that's been surprising is the number of snags we've hit
along the way. As an example, right now Julian is moving some
decorators from canonical.librarian.db into lp.services.database in
order to fix a particular circular import problem. We've easily hit a
dozen, minor issues like this. Where possible, we've tried to fix them
properly.

I've particularly enjoyed seeing obscure, broad doctests become less
obscure, slightly less broad unit tests. We've also added unit tests
for quite a few things that were only tested by accident, although not
as many as I'd like.

This is a big, risky branch. We've been spinning off as much as we
can, but the guts of the change will have to land all at once.

Things to do:
  * There are many XXXs from things that we've noticed along the way.
We ought to fix them.
  * In particular, the interface contract for Builder and
BuildFarmBehavior needs to be updated.
  * The branch needs to be tested big time on dog food.
  * The manager code needs a bit of an overhaul to completely eradicate 
RecordingSlave and the rather silly deferred_list list.
  * buildd-slavescanner needs to be raided by pirates for booty before 
ditching it on the rocks.
  * A few other failing tests.

jml
===========

I'm seriously happy with the work we've done.  It is a little disappointing 
that we didn't finish the work this week, but we vastly underestimated the 
size of the task.  Even after landing a few thousand lines of code already, 
out integration branch is 4k lines long.  This Is Scary.

Thank you Jono for putting a week aside to help.

<send>