launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

Jono and I are sprinting at my house in Worcester, UK, this week to give some 
much needed love to the build mangler^Wmanager.  The main plan is to convert 
its parts that are still synchronous into asynchronous code.

I thought you guys would be interested in our progress, so I'm going to write 
a daily email to this list describing what we've done.

== Day One ==

Today we spent most of the day analysing the code and working out a plan of 
attack.  We also did some exploratory coding to see exactly how many tests 
broke each time we changed something (the answer to that is censored because I 
might swear a lot).

The first change we did was to change BuilderSlave from inheriting 
xmlrpclib.ServerProxy to composition so that we are forced to wrap any calls 
that are made to it.  We want to make all of the methods on BuilderSlave 
eventually return a Deferred, so we started with the "ensurepresent" call. 
These two changes introduced a lot of test fallout, so we took the opportunity 
to write a lot of new unit tests for BuilderSlave and remove some old crappy 
doctest chunks (yay).

We then traced the call stack for "ensurepresent" and made all the code that 
calls it return Deferred.  We fixed the buildd-slave.txt doctest by converting 
a chunk of it to a unit test (we need to use the Trial test case now so 
doctests are right out!), but the sastifaction was short lived because we then 
started looking at buildd-slavescanner.txt which is quite frankly a nightmare.  
It's actually testing binarypackagebuildbehaviour by trying to dispatch builds 
and therefore requiring mock slaves.

At this point we took a step back and had a look at all the other stuff we 
need to fix this week to achieve the ultimate goal of a fully asynchronous 
buildd-manager:

 * Update the rest of BuilderSlave to return Deferreds.
 * Any test that calls a method on the BuilderSlave needs to use the new 
Deferred API
 * Change BuilderSlave to use Twisted XMLRPC
 * Look into Builder.updateStatus and Builder.UpdateBuild (they are currently 
synchronous)
 * Delete the RecordingSlave class and provide some other kind of test double
 * Update all the tests that use RecordingSlave so they work (test_manager 
will be a royal pain!)
 * Review all the usage of slave mocks, and probably delete them.

Tomorrow, we plan to re-write that doctest as a unit test and also fix the 
other behaviour classes so that they work with the new BuilderSlave API.

And now, it's beer o'clock.