launchpad-reviewers team mailing list archive

[Martin Pool <mbp@xxxxxxxxxxxxx>]

Martin Pool has proposed merging lp:~mbp/launchpad/884997-rusage into lp:launchpad.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  Bug #884997 in Launchpad itself: "buildrecipe doesn't report resource usage"
  https://bugs.launchpad.net/launchpad/+bug/884997

For more details, see:
https://code.launchpad.net/~mbp/launchpad/884997-rusage/+merge/80970

This makes buildrecipe log the rusage of bzr dailydeb which will let us see how close to the edge we are wrt memory.

It also fixes some confusing code whereby the argv[0] of buildrecipe is buildrecipe.py.

I have tested this manually by calling in to this code from a python shell.  I would like to write an automatic test but it's a little bit hard because this is a script not a module, and I'm concerned that renaming it will break things in the complicated deployment process.
-- 
https://code.launchpad.net/~mbp/launchpad/884997-rusage/+merge/80970
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~mbp/launchpad/884997-rusage into lp:launchpad.

=== modified file 'lib/canonical/buildd/buildrecipe'
--- lib/canonical/buildd/buildrecipe	2011-02-09 21:10:55 +0000
+++ lib/canonical/buildd/buildrecipe	2011-11-02 01:23:28 +0000
@@ -1,5 +1,5 @@
-#!/usr/bin/env python
-# Copyright 2010 Canonical Ltd.  This software is licensed under the
+#! /usr/bin/env python -u
+# Copyright 2010, 2011 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """A script that builds a package from a recipe and a chroot."""
@@ -12,7 +12,10 @@
 import pwd
 from resource import RLIMIT_AS, setrlimit
 import socket
-from subprocess import call
+from subprocess import (
+    Popen,
+    call,
+    )
 import sys
 
 
@@ -30,6 +33,20 @@
         Exception.__init__(self, 'Not running under Xen.')
 
 
+def call_report_rusage(args):
+    """Run a subprocess.
+
+    Report that it was run, and the resources used, and complain if it fails.
+
+    :return: The process wait status.
+    """
+    print 'RUN %r' % args
+    proc = Popen(args)
+    pid, status, rusage = os.wait4(proc.pid, 0)
+    print(rusage)
+    return status
+
+
 class RecipeBuilder:
     """Builds a package from a recipe."""
 
@@ -92,6 +109,10 @@
         hostname = socket.gethostname()
         bzr_email = 'buildd@%s' % hostname
 
+        print 'Bazaar versions:'
+        check_call(['bzr', 'version'])
+        check_call(['bzr', 'plugins'])
+
         print 'Building recipe:'
         print recipe
         sys.stdout.flush()
@@ -99,7 +120,7 @@
             'DEBEMAIL': self.author_email,
             'DEBFULLNAME': self.author_name.encode('utf-8'),
             'BZR_EMAIL': bzr_email}
-        retcode = call([
+        retcode = call_report_rusage([
             'bzr', 'dailydeb', '--safe', '--no-build', recipe_path,
             self.tree_path, '--manifest', manifest_path,
             '--append-version', '~%s1' % self.distroseries_name], env=env)

=== modified file 'lib/canonical/buildd/sourcepackagerecipe.py'
--- lib/canonical/buildd/sourcepackagerecipe.py	2010-06-30 16:07:47 +0000
+++ lib/canonical/buildd/sourcepackagerecipe.py	2011-11-02 01:23:28 +0000
@@ -86,7 +86,7 @@
         recipe_path = get_chroot_path(self._buildid, 'work/recipe')
         splat_file(recipe_path, self.recipe_text)
         args = [
-            "buildrecipe.py", self._buildid, self.author_name.encode('utf-8'),
+            "buildrecipe", self._buildid, self.author_name.encode('utf-8'),
             self.author_email, self.suite, self.distroseries_name,
             self.component, self.archive_purpose]
         self.runSubProcess(self.build_recipe_path, args)