yellow team mailing list archive

[Gary Poster <gary.poster@xxxxxxxxxxxxx>]

Hi all.

Summary:

*  We have a change in plan on how we are going to connect the slave and 
master.  It should be much simpler than what I've been talking about.  I 
hope Graham and Francesco can get it done in their morning.

*  When we all are working together tomorrow after the morning call, I'd 
like us to focus on actually getting a real LP buildbot slave talking to 
a master.  We want to find out what the problems are, if any, so we have 
a couple of days to fix them before Wednesday.

*  Benji will be out tomorrow.

*  Please help us keep these cards moving, especially if there is a 
handoff at your EoD.

Details:

SLAVE AND MASTER CONNECTION

I talked with all or most of you today about how we could make a very 
dynamic connection between the buildslaves and the build master. 
However, Brad and I really tried to talk through how we would templatize 
the master, based on the current hardcoded lpbuildbot master.cfg, and we 
encountered a conceptual mismatch between juju and buildbot.

In particular, buildbot has an abstraction called a builder.  One or 
more buildslaves can register for a builder to act as a kind of load 
balancing.  Each buildslave for a give builder are expected to be 
essentially identical: this feature is *only* for load balancing.  If 
you want to test different architectures, for instance, you should have 
different builders.  Buildslaves can participate in one or more builders.

Juju only has a concept for buildslaves and build masters.  If we were 
going to try and make the master configuration fully dynamic, we would 
need to enforce a one-to-one relationship between the build slaves and 
the builders.  Brad mentioned that in the future maybe we could have 
subordinate charms for the buildslaves to define builders, but that 
doesn't help us now, even were it to work in the future.

I decided that we should simplify.  Brad and I talked over two options, 
and we settled on the following one.  Line numbers refer to 
http://bazaar.launchpad.net/~launchpad/lpbuildbot/public/view/head:/master.cfg 
.

*  Slaves can specify one or more builders they want to participate in.
*  Master charms need to be able to generate slave names and passwords 
on the fly.
*  Master charms need to substitute in the current slave names and 
passwords into a place like lines 20-21.
* Master charms need to be able to slot in slave names into the builders 
they want to participate in, like lines 179 & 185.  If more than one 
slave participates, this should be comma-separated (and the template 
would need to use 'slavenames' rather than 'slavename' for the key, I 
think, and accept a list).  This part is a bit tricky, because the 
substitution needs to work with the builder names; and because I'm not 
sure what to do if there is no slave.  Maybe it could work like this?

Template:
c['builders'] = [
    {
        'name': "lucid_lp",
        'slavenames': [{slaves-lucid_lp}],
        'builddir': "lucid-devel",
        'factory': launchpad_test_factory,
    },
    {
        'name': "lucid_db_lp",
        'slavenames': [{slaves-lucid_db_lp}],
        'builddir': "lucid-db-devel",
        'factory': launchpad_test_factory,
    },
    ]

I'm not sure if the master will start if slavenames is empty.  We might 
need to do something different because of this.  Maybe we supply a dummy 
slavename, including a name and password registration, until a real one 
shows up?  Maybe we replace/control the entire builder stanza instead? 
 That might be tricky because there are newer and different ways to 
define those stanzas now (buildbot.config.BuilderConfig, in particular). 
 The dummy would be easiest, if ugly...

This is what I hope Graham and Francesco will quickly put together 
Friday morning.  I can dream that it would go that fast. :-)

TESTING THE REAL LP SETUP

I'd like to get to this as soon as possible.  I can imagine a couple of 
unpleasant surprises that might bite us, so let's find out what they are 
so we can fix them before next Wednesday.  If we could get cracking on 
this by Friday, that would be fantastic.  I'll be concerned if we have 
not started this on Monday.

BENJI WILL BE OUT

He sent a handoff note.  Maybe I'll tackle this.  I'd like us to focus 
on what I wrote above, though.

KEEP THE CARDS MOVING

I saw that Francesco's and Graham's card was in the "review" lane by the 
time they had left for their EoD, but I couldn't find a MP for it.  If 
you can keep the cards moving, that will keep us from being blocked, so 
please think about handoff and blockages before you quit for the day.

THAT'S ALL FOLKS

Thanks everyone!  I'm excited by the progress we are making.

Gary