yellow team mailing list archive

[Francesco Banconi <francesco.banconi@xxxxxxxxxxxxx>]

Francesco Banconi has proposed merging lp:~frankban/charms/oneiric/buildbot-slave/use-lpsetup into lp:~yellow/charms/oneiric/buildbot-slave/trunk.

Requested reviews:
  Launchpad Yellow Squad (yellow)

For more details, see:
https://code.launchpad.net/~frankban/charms/oneiric/buildbot-slave/use-lpsetup/+merge/105086

= Summary =

Use lpsetup in place of setuplxc to create the launchpad testing environment
used by buildbot.

== Implementation details ==

Added a new transport, *apt*, that can be used to retrieve and run a script
when the charm install hook is executed.

Renamed the *url* config option to reflect the fact that it can be either
a url or a package name: now it is called, more generically, *source*.

Updated lpbuildbot yaml file to install and run lpsetup.

Fixed a bug preventing extra packages (as listed in yaml config) 
to be installed in the charm instance. 
-- 
https://code.launchpad.net/~frankban/charms/oneiric/buildbot-slave/use-lpsetup/+merge/105086
Your team Launchpad Yellow Squad is requested to review the proposed merge of lp:~frankban/charms/oneiric/buildbot-slave/use-lpsetup into lp:~yellow/charms/oneiric/buildbot-slave/trunk.

=== modified file 'config.yaml'
--- config.yaml	2012-02-16 16:42:57 +0000
+++ config.yaml	2012-05-08 16:39:21 +0000
@@ -1,13 +1,14 @@
 options:
   script-retrieval-method:
     description: |
-      How the script is retrieved. You can use bzr, brz_cat, git,
+      How the script is retrieved. You can use apt, bzr, brz_cat, git,
       mercurial or wget.
     type: string
     default: bzr
-  script-url:
+  script-source:
     description: |
-      The url where this script lives.
+      If script-retrieval-method is "apt", this is the name of the package to
+      install. In the other cases, this is the url where the script lives.
     type: string
   script-path:
     description: |

=== modified file 'examples/lpbuildbot.yaml'
--- examples/lpbuildbot.yaml	2012-03-23 15:48:32 +0000
+++ examples/lpbuildbot.yaml	2012-05-08 16:39:21 +0000
@@ -1,11 +1,11 @@
 buildbot-slave:
   builders: lucid_lp
-  script-retrieval-method: bzr_cat
-  script-url: "http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/utilities/setuplxc.py";
-  script-path: setuplxc.py
+  script-retrieval-method: apt
+  script-source: lpsetup
+  script-path: "/usr/bin/lp-setup"
   # The buildbot user's home directory is /var/lib/buildout.
-  # The LP dependencies will be in /var/lib/buildout/dependencies.
-  script-args: "-u buildbot -e launchpad-pqm@xxxxxxxxxxxxx -f 'Launchpad PQM' -s launchpad_lxc_id_rsa --use-urandom {installdir}"
+  # The LP dependencies will be in /var/lib/buildout/launchpad/dependencies.
+  script-args: "lxc-install -u buildbot -e launchpad-pqm@xxxxxxxxxxxxx -f 'Launchpad PQM' -c {installdir} -S launchpad_lxc_id_rsa --testing"
   extra-repository: deb http://us.archive.ubuntu.com/ubuntu/ lucid main universe
   buildbot-pkg: buildbot/lucid
   extra-packages: testrepository

=== modified file 'hooks/install'
--- hooks/install	2012-04-16 09:23:42 +0000
+++ hooks/install	2012-05-08 16:39:21 +0000
@@ -136,6 +136,11 @@
 bzr = command('bzr')
 
 
+def apt(source, path):
+    apt_get_install(source)
+    return path
+
+
 def bzr_fetch(source, path):
     apt_get_install('bzr')
     target = tempfile.mktemp()
@@ -173,6 +178,7 @@
 
 
 METHODS = {
+    'apt': apt,
     'bzr': bzr_fetch,
     'bzr_cat': bzr_cat,
     'wget': wget,
@@ -181,9 +187,9 @@
     }
 
 
-def handle_script(retrieve, url, path, args):
-    log('Retrieving script: {}.'.format(url))
-    script = retrieve(url, path)
+def handle_script(retrieve, source, path, args):
+    log('Retrieving script: {}.'.format(source))
+    script = retrieve(source, path)
     log('Changing script mode.')
     run('chmod', '+x', script)
     log('Executing script: {}.'.format(script))
@@ -193,7 +199,7 @@
 def main():
     config = get_config()
     method = config.get('script-retrieval-method')
-    url = config.get('script-url')
+    source = config.get('script-source')
     path = config.get('script-path')
     # This is a naive substitution.  We can make it more sophisticated
     # if we discover we need it.  For now, simplicity wins.
@@ -201,6 +207,7 @@
     buildbot_pkg = config.get('buildbot-pkg')
     extra_repo = config.get('extra-repository')
     buildbot_dir = config.get('installdir')
+    extra_pkgs = config.get('extra-packages')
 
     if extra_repo:
         install_extra_repository(extra_repo)
@@ -215,16 +222,20 @@
     run('addgroup', 'buildbot')
     run('usermod', '--gid', 'buildbot', 'buildbot')
 
+    # Installing extra packages if required.
+    if extra_pkgs:
+        apt_get_install(*(pkg.strip() for pkg in extra_pkgs.split()))
+
     config_json.set(config)
 
     retrieve = METHODS.get(method)
-    if retrieve and url and path:
+    if retrieve and source and path:
         # Make buildbot user have a shell by editing /etc/passwd.
         # Otherwise you cannot ssh as this user, which some scripts
         # need (e.g. those that create lxc containers).  We choose sh as
         # a standard and basic "system" shell.
         run('usermod', '-s', '/bin/sh', 'buildbot')
-        sys.exit(handle_script(retrieve, url, path, args))
+        sys.exit(handle_script(retrieve, source, path, args))
 
 
 if __name__ == '__main__':

=== modified file 'revision'
--- revision	2012-03-06 18:23:20 +0000
+++ revision	2012-05-08 16:39:21 +0000
@@ -1,1 +1,1 @@
-8
+9

=== modified file 'tests/buildbot-slave.test'
--- tests/buildbot-slave.test	2012-03-01 16:32:37 +0000
+++ tests/buildbot-slave.test	2012-05-08 16:39:21 +0000
@@ -82,7 +82,7 @@
             options = {}
         options.update({
             'script-retrieval-method': 'bzr_cat',
-            'script-url':
+            'script-source':
                 'http://bazaar.launchpad.net/~yellow/charms/'
                 'oneiric/buildbot-slave/trunk/tests/create_file.py',
             'script-path': 'create_file.py',