yellow team mailing list archive

[Gary Poster <gary.poster@xxxxxxxxxxxxx>]

Hi.  We are getting hardware from RT #53012 to support the Launchpad
parallel tests.  This RT is about configuring them, and the buildbot
master, for the tests.

This is also closely related to the work Liam did for RT #50242.  We
have rewritten the tool we use to configure machines, but the basic
requirements are the same.  We have a few changes from what he set up
then, based on yet more experience with LXC and the Launchpad tests, but
the practical differences should be fairly small.

I can offer two options for this, and I'm happy to hear other
suggestions and requests.

First, and my preferred option, we could try again to leverage work we
have done so that the tools you use to configure the machines and the
tools we use to configure the machines are as alike as possible.

My understanding from before was that the host computer needed to be
configured via Puppet and GSAs; but that Webops could have free reign in
the LXC containers even to the point of letting them run as root there.
 If that still stands, this approach could work nicely, I think.

We are in the process of refactoring lpsetup, which is a debian-packaged
set of tools that take the place of setuplxc, the script that Liam
reviewed and consulted before.  lpsetup is here:
https://code.launchpad.net/~canonical-launchpad-branches/lpsetup/trunk .

We are refactoring it for a number of reasons, but not least to try and
make parts of the tool reusable by webops.

One part of the tool is designed to be run with root privileges on the
host.  You would never run this command, replacing it with mimicking its
responsibilities in Puppet (and possibly other changes done via GSAs).
This would look very similar to what Liam set up before: webops would be
able to run as the buildbot user, and the buildbot user would have sudo
privileges to run a small set of scripts that should be easy to review
for security.  webops would have root privileges in the lxc container
that we created.

Webops would be able to use the lpsetup tool as the buildbot user or as
the container's root user to do everything else.  Specifically, they
would use three commands, initially and periodically in the future.

In the container as root, they would run an "init" command to initialize
the container.  (They would run this command, or another one associated
with it, to update the lxc container in the future.)

In the host or the container as the buildbot user, they would run a
"get" command to get the Launchpad tree.

In the container as the buildbot user, they would run an "update"
command to get the Launchpad tree's dependencies.

In the container as root, they might have to run additional commands
(launchpad-database-setup and make install for instance) but ideally
eventually these will be rolled into the "init" command.  Also,
importantly, you would be doing whatever the developers would be doing,
now and in the future.

If this sounds acceptable, I'll list what we need to do in the host as
root, and/or work with someone in webops to figure out the diff between
what was done for RT #50242 and what we need now; and then we'll also
continue fixing up lpsetup so you all can use it, trying to be ready as
fast as possible.

If that's not OK, our other approach be to just share lpsetup with you
and have you migrate it to whatever format you need.

I'm sure there's middle ground between those two, as well as entirely
different approaches.  An obvious middle-ground approach is that you
only allow the tool to be run as the buildbot user, never as root even
in the container.  That would be a shame, but better than reusing nothing.

Please let us know what you think.

Thanks

Gary