launchpad-dev team mailing list archive

[Michael Hudson <michael.hudson@xxxxxxxxxxxxx>]

Hi all,

I'm at the end of my stint as the build engineer, so it makes sense to
send an email talking about what I have and haven't done.

I've mostly focused on the EC2-using infrastructure: ec2test and
buildbot.  Here's some highlights:

 * I wrote https://dev.launchpad.net/Trunk/Glue describing how our 4
   trunks and buildbot set up is put together.

 * I added a 'force build' feature to buildbot.

 * I set up a buildbot builder running make jscheck daily.

 * I, or mostly Tom really, set up a staging buildbot that we can use
   to test config changes and system upgrades.

 * I built a buildbot package that's based on a release of buildbot,
   which seems to be running OK on staging (too scared to try it on
   production in week 4).

 * I drastically refactored ec2test into a much more organized 'ec2'
   command with various subcommands.

 * This ec2 tool has a command to update the ec2 AMI we use in one
   step (ec2 update-image).

 * I rejigged things to make ec2test --headless go headless much
   faster.

 * I extended ec2 to be able to start with a blank AMI such as those
   provided by Eric Hammond at http://alestic.com/ and then either
   bundle it or run tests on it.

   * We should be able to use this to start running the tests on lucid
     as soon as AMIs become available PPA's start supporting it -- a
     nice, mostly unintentional, side effect.

 * I extended and used ec2 update-image to make an AMI that can be
   made publically available next week.

   * I haven't made it publically available yet, because while the
     image works with the trunk ec2, it is rather slow to start up.

 * Away from the EC2 specific stuff, I triaged and commented on lots
   of build-infrastructure bugs.

   * In particular, https://dev.launchpad.net/FasterTests now has
     links to and from various specific bug reports.

General comments about the Build Engineer role:

 * I think the rotating build engineer role is a good idea.

 * The amount of work to be done in the area is quite daunting, and
   many of the problems are hard in frustrating niggly ways.

 * It's hard to get things done on the infrastructure in week 4!

 * In addition, not being able to land branches in week 4 is a pain,
   even more than normal.  As the build engineer's work is sort of
   sideways to the main thrust of launchpad development, I've acquired
   a stack of interdependent branches that I can't land until Monday
   :/

 * A cycle is probably about the right length of time.

 * My main 'dashboard' has been the list of bugs tagged with
   'build-infrastructure':

   https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=build-infrastructure

   I think this is a good way to work and I'd recommend it to those
   who follow me.

So in summary, I think I've had a reasonably productive cycle, and
best of luck to Gavin in his stint as Build Engineer!

Cheers,
mwh