launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #05831
[Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #899972 in Launchpad itself: "cron.germinate is very slow"
https://bugs.launchpad.net/launchpad/+bug/899972
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/refactor-cron-germinate/+merge/84624
= Summary =
cron.germinate currently takes on the order of ten minutes, and really doesn't have a particularly good excuse for taking so long. Fixing this would probably be enough to let us move back to 30-minute publisher cycles for Ubuntu (as we used to have with dak, back in the dawn of time), which would increase our velocity and make some of us very happy.
The main reason it's so slow is that it runs germinate as a separate process once for each of eight flavours (Ubuntu, Kubuntu, etc., each with its own seed collection) on each of five architectures. There are a few inefficiencies inherent in this approach, but the most significant is that it has to expand dependencies and build-dependencies of the seeds that are common to all the flavours eight times as often as necessary. Since the build-dependency chain in particular of the base system winds its way through a good fraction of main, this winds up being rather a lot of duplicated work.
== Proposed fix ==
I've been working on this problem for some time now, and have just released germinate 2.0 to support solving it properly. The most important change here is that germinate can now process multiple seed collections for a single architecture in a single instance of the core Germinator class, which allows reusing the expansions of common seeds. While I could technically have extended the command-line interface further for this, I felt that the command-line interface was already far too complicated, and decided instead to export a public, documented, and stable Python interface which can be used for this purpose.
This branch is the other half of that fix: a LaunchpadScript that uses germinate's new Python interface to do the same work as the top part of cron.germinate (i.e. everything except for maintenance-check.py). A stripped-down subset of this with just the logic (germinating seeds and producing output) runs in about three minutes on my laptop.
== Pre-implementation notes ==
The Launchpad position for a while has been that cron.germinate is owned by Ubuntu Engineering, and indeed the vast bulk of the work here was in preparing a new germinate release in Ubuntu that does what we need. However, I have talked with various Launchpad people over the last few months so that it was clear that I was working on this, and I did need some help figuring out how to approach testing. Initially I'd been planning on doing a full Soyuz publisher run so that I had Packages and Sources files to work with, but Julian said this would be too slow and pointed me towards methods like factory.makeSourcePackagePublishingHistory, which allow me to use getIndexStanza et al so that I don't have to write out RFC822 data by hand.
== Implementation details ==
I had to add a new attribute on PublisherConfig pointing to the germinate output directory; previously this was only known to the shell script.
I added a 'generate_extra_overrides' user to security.cfg, which seemed to be in line with other archivepublisher scripts. Was this the right approach? Does it require any special deployment steps?
The actual override generation was a fairly straightforward translation from shell to Python.
Predictably, the bulk of the work was in writing tests, since cron.germinate was previously untested and I didn't think continuing with untested code was likely to fly once it was in Python. The top-level shell script is still untested, but there's a lot less in it and I expect that it can eventually be removed entirely.
I ran into a couple of existing bugs that slowed me down. Firstly, I'd hoped to use the fake librarian to speed things up a bit, but ran into bug 713764, which was beyond my ability to fix (although I tried). Secondly, bug 694140 seriously confused me for a while; I added much the same workaround as is found elsewhere in Launchpad for that problem so that germinate's log output doesn't get lost.
I wanted to include a test that the cronscript could be run standalone, but I couldn't see how to ensure that a suitable PublisherConfig was in place.
== Tests ==
bin/test -vvct generate_extra_overrides
This branch makes some small changes to other archivepublisher tests, so:
bin/test -vvct archivepublisher
== Demo and Q/A ==
The deployment steps listed in bug 899972 and https://rt.admin.canonical.com/Ticket/Display.html?id=49745 need to be completed before we can land this branch.
Once that's done, we can Q/A this by doing a timed control publisher run on mawson, rolling out this change, taking a backup copy of ubuntu-germinate and ubuntu-misc, and then doing another timed publisher run and comparing the ubuntu-germinate and ubuntu-misc directories before and after. (There may be some harmless ordering changes and such.)
== Lint ==
./cronscripts/generate-extra-overrides.py
8: '_pythonpath' imported but unused
Seems to be standard practice in cronscripts.
./lib/lp/archivepublisher/tests/publisher-config.txt
1: narrative uses a moin header.
26: narrative uses a moin header.
46: narrative uses a moin header.
114: narrative uses a moin header.
138: narrative uses a moin header.
163: narrative uses a moin header.
178: want exceeds 78 characters.
179: want exceeds 78 characters.
180: want exceeds 78 characters.
182: want exceeds 78 characters.
This was pre-existing lint; I only made the minimal changes necessary to add the 'germinateroot' attribute, which haven't fundamentally made the situation any worse.
--
https://code.launchpad.net/~cjwatson/launchpad/refactor-cron-germinate/+merge/84624
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad.
=== added file 'cronscripts/generate-extra-overrides.py'
--- cronscripts/generate-extra-overrides.py 1970-01-01 00:00:00 +0000
+++ cronscripts/generate-extra-overrides.py 2011-12-06 15:03:58 +0000
@@ -0,0 +1,18 @@
+#!/usr/bin/python -S
+#
+# Copyright 2011 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Generate extra overrides using Germinate."""
+
+import _pythonpath
+
+from lp.archivepublisher.scripts.generate_extra_overrides import (
+ GenerateExtraOverrides,
+ )
+
+
+if __name__ == '__main__':
+ script = GenerateExtraOverrides(
+ "generate-extra-overrides", dbuser='generate_extra_overrides')
+ script.lock_and_run()
=== modified file 'cronscripts/publishing/cron.germinate'
--- cronscripts/publishing/cron.germinate 2011-10-27 11:36:13 +0000
+++ cronscripts/publishing/cron.germinate 2011-12-06 15:03:58 +0000
@@ -12,8 +12,12 @@
GERMINATEROOT=$ARCHIVEROOT/../ubuntu-germinate
LAUNCHPADROOT=${TEST_LAUNCHPADROOT:-/srv/launchpad.net/codelines/current}
+GENERATE=$LAUNCHPADROOT/cronscripts/generate-extra-overrides.py
MAINTAINCE_CHECK=$LAUNCHPADROOT/cronscripts/publishing/maintenance-check.py
+FLAVOURS="ubuntu kubuntu kubuntu-mobile edubuntu xubuntu mythbuntu lubuntu"
+FLAVOURS="$FLAVOURS ubuntustudio"
+
## Check to see if another germinate run is in progress
LOCKFILE=$LOCKROOT/cron.germinate.lock
@@ -28,134 +32,9 @@
trap cleanup EXIT
-suite=`$LAUNCHPADROOT/scripts/ftpmaster-tools/lp-query-distro.py development`
-archs=`$LAUNCHPADROOT/scripts/ftpmaster-tools/lp-query-distro.py -s "$suite" archs`
-
-echo -n "Running germinate... "
cd $GERMINATEROOT
-# Clean up temporary files
-rm -f \
- germinate.output ALL ALL.sources \
- UBUNTU-* KUBUNTU-* EDUBUNTU-* XUBUNTU-* MYTHBUNTU-* LUBUNTU-* \
- UBUNTUSTUDIO-*
-rm -f all_* all.sources_*
-rm -rf dists
-
-# Grab local copies of Sources and Packages files, to avoid problems in case
-# the archive changes under our feet.
-for component in main universe restricted multiverse; do
- base="dists/$suite/$component"
- mkdir -p "$base/source"
- zcat "$ARCHIVEROOT/$base/source/Sources.gz" > "$base/source/Sources"
- for arch in $archs; do
- mkdir -p "$base/binary-$arch" "$base/debian-installer/binary-$arch"
- zcat "$ARCHIVEROOT/$base/binary-$arch/Packages.gz" \
- > "$base/binary-$arch/Packages"
- zcat "$ARCHIVEROOT/$base/debian-installer/binary-$arch/Packages.gz" \
- > "$base/debian-installer/binary-$arch/Packages"
- done
-done
-
-> "$MISCROOT/more-extra.override.$suite.main.new"
-
-germinate_components=main,universe,restricted,multiverse
-for distro in \
- ubuntu kubuntu kubuntu-mobile edubuntu xubuntu mythbuntu lubuntu \
- ubuntustudio
-do
- DISTRO="$(echo $distro | tr a-z A-Z)"
- germinate_suite="$distro.$suite"
- for arch in $archs; do
- # Run germinate
- echo " **************** $distro/$suite/$arch ********************* " >> germinate.output
- germinate \
- --no-rdepends \
- -m "file://$(pwd)/" \
- -s "$germinate_suite" \
- -d "$suite" \
- -c "$germinate_components" \
- -a $arch >> germinate.output 2>&1
-
- # The structure file is generally useful; keep per distro/suite/arch
- # copies for convenience
- cp structure structure_"$distro"_"$suite"_"$arch"
-
- # Keep per distro/suite/arch copies of 'all' and 'all.sources' for
- # anastacia.
- cp all all_"$distro"_"$suite"_"$arch"
- cp all.sources all.sources_"$distro"_"$suite"_"$arch"
-
- # Keep per distro/suite/arch copies of 'minimal' and 'standard' for
- # jessica.
- cp minimal minimal_"$distro"_"$suite"_"$arch"
- cp standard standard_"$distro"_"$suite"_"$arch"
-
- # Keep amalgamated copies of 'all' and 'all.sources', just for convenience
- cat all >> ALL; cat all.sources >> ALL.sources
-
- # We need to fetch a number of seeds so that we can generate Task fields
- # for them. This changes over time and differs from derivative to
- # derivative, so it's best to just fetch them all.
- taskseeds="$(cut -d: -f1 structure | xargs -n1 | sort -u)"
-
- for seed in $taskseeds; do
- cp "$seed" "$seed"_"$distro"_"$suite"_"$arch"
- done
- echo " ********************************************************************** " >> germinate.output
- echo "" >> germinate.output
- echo -n "."
-
- ## Generate apt-ftparchive 'extra' overrides for Task: fields
- for seed in $taskseeds; do
- if ! grep -iq '^Task-' "$seed.seedtext"; then
- continue
- fi
- # If the seed contains Task-Name header, override the normal behavior
- if grep -iq '^Task-Name:' "$seed.seedtext"; then
- task=$(grep '^Task-Name:' "$seed.seedtext" | cut -d: -f2)
- elif grep -iq '^Task-Per-Derivative:' "$seed.seedtext"; then
- task="$distro-$seed"
- else
- # If a seed is not per-derivative, then we only honour it for Ubuntu,
- # and its task name is archive-global.
- if [ "$distro" = ubuntu ]; then
- task="$seed"
- else
- continue
- fi
- fi
- if grep -iq '^Task-Seeds:' "$seed.seedtext"; then
- scanseeds="$( (grep '^Task-Seeds:' "$seed.seedtext" | cut -d: -f2 | xargs -n1; echo "$seed") | sort -u )"
- else
- scanseeds="$seed"
- fi
- for scanseed in $scanseeds; do
- egrep -v -- \
- "^(-|Package| )" "$scanseed"_"$distro"_"$suite"_"$arch" |
- awk '{print $1}' |
- sed -e "s,$,/$arch Task $task," |
- sort -u >> "$MISCROOT/more-extra.override.$suite.main.new"
- done
- done
-
- # Generate apt-ftparchive 'extra' overrides for Build-Essential: fields
- if [ -e build-essential ] && [ "$distro" = ubuntu ]; then
- # Keep a copy, just for convenience
- cp build-essential build-essential_"$distro"_"$suite"_"$arch"
- egrep -v -- \
- "^(-|Package| )" build-essential_"$distro"_"$suite"_"$arch" |
- awk '{print $1}' |
- sed -e "s,$,/$arch Build-Essential yes," |
- sort -u >> "$MISCROOT/more-extra.override.$suite.main.new"
- fi
- done
-done
-echo " done."
-
-mv -f \
- "$MISCROOT/more-extra.override.$suite.main.new" \
- "$MISCROOT/more-extra.override.$suite.main"
+$GENERATE -d ubuntu $FLAVOURS
# Now generate the Supported extra overrides for all supported distros.
SUITES=`$LAUNCHPADROOT/scripts/ftpmaster-tools/lp-query-distro.py supported`
=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg 2011-12-06 11:00:07 +0000
+++ database/schema/security.cfg 2011-12-06 15:03:58 +0000
@@ -2264,6 +2264,10 @@
type=user
groups=archivepublisher
+[generate_extra_overrides]
+type=user
+groups=archivepublisher
+
[process_death_row]
type=user
groups=archivepublisher
=== modified file 'lib/lp/archivepublisher/config.py'
--- lib/lp/archivepublisher/config.py 2011-08-29 16:43:10 +0000
+++ lib/lp/archivepublisher/config.py 2011-12-06 15:03:58 +0000
@@ -74,10 +74,12 @@
pubconf.overrideroot = pubconf.archiveroot + '-overrides'
pubconf.cacheroot = pubconf.archiveroot + '-cache'
pubconf.miscroot = pubconf.archiveroot + '-misc'
+ pubconf.germinateroot = pubconf.archiveroot + '-germinate'
else:
pubconf.overrideroot = None
pubconf.cacheroot = None
pubconf.miscroot = None
+ pubconf.germinateroot = None
pubconf.poolroot = os.path.join(pubconf.archiveroot, 'pool')
pubconf.distsroot = os.path.join(pubconf.archiveroot, 'dists')
@@ -106,6 +108,7 @@
self.cacheroot,
self.overrideroot,
self.miscroot,
+ self.germinateroot,
self.temproot,
]
=== added file 'lib/lp/archivepublisher/scripts/generate_extra_overrides.py'
--- lib/lp/archivepublisher/scripts/generate_extra_overrides.py 1970-01-01 00:00:00 +0000
+++ lib/lp/archivepublisher/scripts/generate_extra_overrides.py 2011-12-06 15:03:58 +0000
@@ -0,0 +1,292 @@
+# Copyright 2011 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Generate extra overrides using Germinate."""
+
+__metaclass__ = type
+__all__ = [
+ 'GenerateExtraOverrides',
+ ]
+
+import logging
+from optparse import OptionValueError
+import os
+
+from germinate.germinator import Germinator
+from germinate.archive import TagFile
+from germinate.log import GerminateFormatter
+from germinate.seeds import SeedStructure
+
+from zope.component import getUtility
+
+from canonical.launchpad.webapp.dbpolicy import (
+ DatabaseBlockedPolicy,
+ SlaveOnlyDatabasePolicy,
+ )
+from lp.archivepublisher.config import getPubConfig
+from lp.registry.interfaces.distribution import IDistributionSet
+from lp.registry.interfaces.series import SeriesStatus
+from lp.services.scripts.base import (
+ LaunchpadScript,
+ LaunchpadScriptFailure,
+ )
+from lp.services.utils import file_exists
+from lp.soyuz.enums import ArchivePurpose
+
+
+class AtomicFile:
+ """Facilitate atomic writing of files."""
+
+ def __init__(self, filename):
+ self.filename = filename
+ self.fd = open('%s.new' % self.filename, 'w')
+
+ def __enter__(self):
+ return self.fd
+
+ def __exit__(self, exc_type, exc_value, exc_tb):
+ self.fd.close()
+ os.rename('%s.new' % self.filename, self.filename)
+
+
+class GenerateExtraOverrides(LaunchpadScript):
+ """Main class for scripts/ftpmaster-tools/generate-task-overrides.py."""
+
+ def __init__(self, *args, **kwargs):
+ super(GenerateExtraOverrides, self).__init__(*args, **kwargs)
+ self.germinate_logger = None
+
+ def add_my_options(self):
+ """Add a 'distribution' context option."""
+ self.parser.add_option(
+ '-d', '--distribution', dest='distribution',
+ help='Context distribution name.')
+
+ @property
+ def name(self):
+ """See `LaunchpadScript`."""
+ # Include distribution name. Clearer to admins, but also
+ # puts runs for different distributions under separate
+ # locks so that they can run simultaneously.
+ return "%s-%s" % (self._name, self.options.distribution)
+
+ def processOptions(self):
+ """Handle command-line options."""
+ if self.options.distribution is None:
+ raise OptionValueError("Specify a distribution.")
+
+ self.distribution = getUtility(IDistributionSet).getByName(
+ self.options.distribution)
+ if self.distribution is None:
+ raise OptionValueError(
+ "Distribution '%s' not found." % self.options.distribution)
+
+ series = None
+ wanted_status = (SeriesStatus.DEVELOPMENT,
+ SeriesStatus.FROZEN)
+ for status in wanted_status:
+ series = self.distribution.getSeriesByStatus(status)
+ if series.count() > 0:
+ break
+ else:
+ raise LaunchpadScriptFailure(
+ 'There is no DEVELOPMENT distroseries for %s' %
+ self.options.distribution)
+ self.series = series[0]
+
+ # Even if DistroSeries.component_names starts including partner, we
+ # don't want it; this applies to the primary archive only.
+ self.components = [component
+ for component in self.series.component_names
+ if component != 'partner']
+
+ def getConfig(self):
+ """Set up a configuration object for this archive."""
+ for archive in self.distribution.all_distro_archives:
+ # We only work on the primary archive.
+ if archive.purpose == ArchivePurpose.PRIMARY:
+ return getPubConfig(archive)
+ else:
+ raise LaunchpadScriptFailure(
+ 'There is no PRIMARY archive for %s' %
+ self.options.distribution)
+
+ def setUpDirs(self):
+ """Create output directories if they did not already exist."""
+ germinateroot = self.config.germinateroot
+ if not file_exists(germinateroot):
+ self.logger.debug("Creating germinate root %s.", germinateroot)
+ os.makedirs(germinateroot)
+ miscroot = self.config.miscroot
+ if not file_exists(miscroot):
+ self.logger.debug("Creating misc root %s.", miscroot)
+ os.makedirs(miscroot)
+
+ def addLogHandler(self):
+ """Send germinate's log output to a separate file."""
+ if self.germinate_logger is not None:
+ return
+
+ self.germinate_logger = logging.getLogger('germinate')
+ self.germinate_logger.setLevel(logging.INFO)
+ log_file = os.path.join(self.config.germinateroot, 'germinate.output')
+ handler = logging.FileHandler(log_file, mode='w')
+ handler.setFormatter(GerminateFormatter())
+ self.germinate_logger.addHandler(handler)
+ self.germinate_logger.propagate = False
+
+ def setUp(self):
+ """Process options, and set up internal state."""
+ self.processOptions()
+ self.config = self.getConfig()
+ self.setUpDirs()
+ self.addLogHandler()
+
+ def makeSeedStructures(self, series_name, flavours, seed_bases=None):
+ structures = {}
+ for flavour in flavours:
+ structures[flavour] = SeedStructure(
+ '%s.%s' % (flavour, series_name), seed_bases=seed_bases)
+ return structures
+
+ def outputPath(self, flavour, series_name, arch, base):
+ return os.path.join(
+ self.config.germinateroot,
+ '%s_%s_%s_%s' % (base, flavour, series_name, arch))
+
+ def runGerminate(self, override_file, series_name, arch, flavours,
+ structures):
+ germinator = Germinator(arch)
+
+ # Read archive metadata.
+ archive = TagFile(
+ series_name, self.components, arch,
+ 'file://%s' % self.config.archiveroot, cleanup=True)
+ germinator.parse_archive(archive)
+
+ for flavour in flavours:
+ self.logger.info('Germinating for %s/%s/%s',
+ flavour, series_name, arch)
+ # Add this to the germinate log as well so that that can be
+ # debugged more easily. Log a separator line first.
+ self.germinate_logger.info('', extra={'progress': True})
+ self.germinate_logger.info('Germinating for %s/%s/%s',
+ flavour, series_name, arch,
+ extra={'progress': True})
+
+ # Expand dependencies.
+ structure = structures[flavour]
+ germinator.plant_seeds(structure)
+ germinator.grow(structure)
+ germinator.add_extras(structure)
+
+ # Write output files.
+
+ # The structure file makes it possible to figure out how the
+ # other output files relate to each other.
+ structure.write(self.outputPath(
+ flavour, series_name, arch, 'structure'))
+
+ # "all" and "all.sources" list the full set of binary and source
+ # packages respectively for a given flavour/suite/architecture
+ # combination.
+ all_path = self.outputPath(flavour, series_name, arch, 'all')
+ all_sources_path = self.outputPath(
+ flavour, series_name, arch, 'all.sources')
+ germinator.write_all_list(structure, all_path)
+ germinator.write_all_source_list(structure, all_sources_path)
+
+ # Write the dependency-expanded output for each seed. Several
+ # of these are used by archive administration tools, and others
+ # are useful for debugging, so it's best to just write them all.
+ for seedname in structure.names:
+ germinator.write_full_list(
+ structure,
+ self.outputPath(flavour, series_name, arch, seedname),
+ seedname)
+
+ def writeOverrides(seedname, key, value):
+ packages = germinator.get_full(structure, seedname)
+ for package in sorted(packages):
+ print >>override_file, '%s/%s %s %s' % (
+ package, arch, key, value)
+
+ # Generate apt-ftparchive "extra overrides" for Task fields.
+ for seedname in structure.names:
+ if seedname == 'extra':
+ continue
+
+ task_headers = {}
+ with structure[seedname] as seedtext:
+ for line in seedtext:
+ if line.lower().startswith('task-') and ':' in line:
+ key, value = line.split(':', 1)
+ # e.g. "Task-Name" => "name"
+ key = key[5:].lower()
+ task_headers[key] = value.strip()
+ if not task_headers:
+ continue
+
+ # Work out the name of the Task to be generated from this
+ # seed. If there is a Task-Name header, it wins; otherwise,
+ # seeds with a Task-Per-Derivative header are honoured for
+ # all flavours and put in an appropriate namespace, while
+ # other seeds are only honoured for the first flavour and
+ # have archive-global names.
+ if 'name' in task_headers:
+ task = task_headers['name']
+ elif 'per-derivative' in task_headers:
+ task = '%s-%s' % (flavour, seedname)
+ elif flavour == flavours[0]:
+ task = seedname
+ else:
+ continue
+
+ # The list of packages in this task come from this seed plus
+ # any other seeds listed in a Task-Seeds header.
+ scan_seeds = set([seedname])
+ if 'seeds' in task_headers:
+ scan_seeds.update(task_headers['seeds'].split())
+ for scan_seed in sorted(scan_seeds):
+ writeOverrides(scan_seed, 'Task', task)
+
+ # Generate apt-ftparchive "extra overrides" for Build-Essential
+ # fields.
+ if ('build-essential' in structure.names and
+ flavour == flavours[0]):
+ writeOverrides('build-essential', 'Build-Essential', 'yes')
+
+ def generateExtraOverrides(self, series_name, series_architectures,
+ flavours, seed_bases=None):
+ structures = self.makeSeedStructures(
+ series_name, flavours, seed_bases=seed_bases)
+
+ override_path = os.path.join(
+ self.config.miscroot,
+ 'more-extra.override.%s.main' % series_name)
+ with AtomicFile(override_path) as override_file:
+ for arch in series_architectures:
+ self.runGerminate(
+ override_file, series_name, arch, flavours, structures)
+
+ def process(self, seed_bases=None):
+ """Do the bulk of the work."""
+ self.setUp()
+
+ series_name = self.series.name
+ series_architectures = sorted(
+ [arch.architecturetag for arch in self.series.architectures])
+
+ # This takes a while. Ensure that we do it without keeping a
+ # database transaction open.
+ self.txn.commit()
+ with DatabaseBlockedPolicy():
+ self.generateExtraOverrides(
+ series_name, series_architectures, self.args,
+ seed_bases=seed_bases)
+
+ def main(self):
+ """See `LaunchpadScript`."""
+ # This code has no need to alter the database.
+ with SlaveOnlyDatabasePolicy():
+ self.process()
=== modified file 'lib/lp/archivepublisher/tests/publisher-config.txt'
--- lib/lp/archivepublisher/tests/publisher-config.txt 2010-10-17 13:35:20 +0000
+++ lib/lp/archivepublisher/tests/publisher-config.txt 2011-12-06 15:03:58 +0000
@@ -14,6 +14,7 @@
... 'overrideroot',
... 'cacheroot',
... 'miscroot',
+ ... 'germinateroot',
... 'temproot',
... ]
@@ -38,6 +39,7 @@
overrideroot: /var/tmp/archive/ubuntutest-overrides
cacheroot: /var/tmp/archive/ubuntutest-cache
miscroot: /var/tmp/archive/ubuntutest-misc
+ germinateroot: /var/tmp/archive/ubuntutest-germinate
temproot: /var/tmp/archive/ubuntutest-temp
@@ -80,6 +82,7 @@
overrideroot: None
cacheroot: None
miscroot: None
+ germinateroot: None
temproot: /var/tmp/archive/ubuntutest-temp
There is a separate location for private PPAs that is used if the
@@ -108,6 +111,7 @@
overrideroot: None
cacheroot: None
miscroot: None
+ germinateroot: None
temproot: /var/tmp/archive/ubuntutest-temp
@@ -131,6 +135,7 @@
overrideroot: None
cacheroot: None
miscroot: None
+ germinateroot: None
temproot: /var/tmp/archive/ubuntutest-temp
@@ -155,6 +160,7 @@
overrideroot: None
cacheroot: None
miscroot: None
+ germinateroot: None
temproot: /var/tmp/archive/ubuntutest-temp
@@ -177,4 +183,5 @@
overrideroot: /var/tmp/archive/ubuntutest-rebuildtest99/ubuntutest-overrides
cacheroot: /var/tmp/archive/ubuntutest-rebuildtest99/ubuntutest-cache
miscroot: /var/tmp/archive/ubuntutest-rebuildtest99/ubuntutest-misc
+ germinateroot: /var/tmp/archive/ubuntutest-rebuildtest99/ubuntutest-germinate
temproot: /var/tmp/archive/ubuntutest-rebuildtest99/ubuntutest-temp
=== added file 'lib/lp/archivepublisher/tests/test_generate_extra_overrides.py'
--- lib/lp/archivepublisher/tests/test_generate_extra_overrides.py 1970-01-01 00:00:00 +0000
+++ lib/lp/archivepublisher/tests/test_generate_extra_overrides.py 2011-12-06 15:03:58 +0000
@@ -0,0 +1,565 @@
+# Copyright 2011 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test for the `generate-extra-overrides` script."""
+
+__metaclass__ = type
+
+import logging
+from optparse import OptionValueError
+import os
+import tempfile
+
+from germinate import (
+ archive,
+ germinator,
+ seeds,
+ )
+
+import transaction
+
+from canonical.testing.layers import (
+ LaunchpadZopelessLayer,
+ ZopelessDatabaseLayer,
+ )
+from lp.archivepublisher.scripts.generate_extra_overrides import (
+ AtomicFile,
+ GenerateExtraOverrides,
+ )
+from lp.archivepublisher.utils import RepositoryIndexFile
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.registry.interfaces.series import SeriesStatus
+from lp.services.log.logger import DevNullLogger
+from lp.services.osutils import (
+ ensure_directory_exists,
+ open_for_writing,
+ )
+from lp.services.scripts.base import LaunchpadScriptFailure
+from lp.services.utils import file_exists
+from lp.soyuz.enums import PackagePublishingStatus
+from lp.testing import TestCaseWithFactory
+from lp.testing.faketransaction import FakeTransaction
+
+
+def file_contents(path):
+ """Return the contents of the file at path."""
+ with open(path) as handle:
+ return handle.read()
+
+
+class TestAtomicFile(TestCaseWithFactory):
+ """Tests for the AtomicFile helper class."""
+
+ layer = ZopelessDatabaseLayer
+
+ def test_atomic_file_creates_file(self):
+ # AtomicFile creates the named file with the requested contents.
+ self.useTempDir()
+ filename = self.factory.getUniqueString()
+ text = self.factory.getUniqueString()
+ with AtomicFile(filename) as test:
+ test.write(text)
+ self.assertEqual(text, file_contents(filename))
+
+ def test_atomic_file_removes_dot_new(self):
+ # AtomicFile does not leave .new files lying around.
+ self.useTempDir()
+ filename = self.factory.getUniqueString()
+ with AtomicFile(filename):
+ pass
+ self.assertFalse(file_exists("%s.new" % filename))
+
+
+class TestGenerateExtraOverrides(TestCaseWithFactory):
+ """Tests for the actual `GenerateExtraOverrides` script."""
+
+ layer = LaunchpadZopelessLayer
+
+ def setUp(self):
+ super(TestGenerateExtraOverrides, self).setUp()
+ self.seeddir = self.makeTemporaryDirectory()
+ # XXX cjwatson 2011-12-06 bug=694140: Make sure germinate doesn't
+ # lose its loggers between tests, due to Launchpad's messing with
+ # global log state.
+ archive._logger = logging.getLogger('germinate.archive')
+ germinator._logger = logging.getLogger('germinate.germinator')
+ seeds._logger = logging.getLogger('germinate.seeds')
+
+ def assertFilesEqual(self, expected_path, observed_path):
+ self.assertEqual(
+ file_contents(expected_path), file_contents(observed_path))
+
+ def makeDistro(self):
+ """Create a distribution for testing.
+
+ The distribution will have a root directory set up, which will
+ be cleaned up after the test. It will have an attached archive.
+ """
+ return self.factory.makeDistribution(
+ publish_root_dir=unicode(self.makeTemporaryDirectory()))
+
+ def makeScript(self, distribution, run_setup=True, flavours=None):
+ """Create a script for testing."""
+ if distribution is None:
+ distribution = self.makeDistro()
+ test_args = ['-d', distribution.name]
+ if flavours is not None:
+ test_args.extend(flavours)
+ script = GenerateExtraOverrides(test_args=test_args)
+ script.logger = DevNullLogger()
+ script.txn = FakeTransaction()
+ if run_setup:
+ script.setUp()
+ else:
+ script.distribution = distribution
+ return script
+
+ def makePackage(self, component, dases, **kwargs):
+ """Create a published source and binary package for testing."""
+ package = self.factory.makeDistributionSourcePackage(
+ distribution=dases[0].distroseries.distribution)
+ spph = self.factory.makeSourcePackagePublishingHistory(
+ distroseries=dases[0].distroseries,
+ pocket=PackagePublishingPocket.RELEASE,
+ status=PackagePublishingStatus.PUBLISHED,
+ sourcepackagename=package.name, component=component)
+ for das in dases:
+ build = self.factory.makeBinaryPackageBuild(
+ source_package_release=spph.sourcepackagerelease,
+ distroarchseries=das, processor=das.default_processor)
+ bpr = self.factory.makeBinaryPackageRelease(
+ binarypackagename=package.name, build=build,
+ component=component, **kwargs)
+ lfa = self.factory.makeLibraryFileAlias(
+ filename='%s.deb' % package.name)
+ transaction.commit()
+ bpr.addFile(lfa)
+ self.factory.makeBinaryPackagePublishingHistory(
+ binarypackagerelease=bpr, distroarchseries=das,
+ pocket=PackagePublishingPocket.RELEASE,
+ status=PackagePublishingStatus.PUBLISHED)
+ return package
+
+ def makeIndexFiles(self, script, distroseries):
+ """Create a limited subset of index files for testing."""
+ ensure_directory_exists(script.config.temproot)
+
+ for component in distroseries.components:
+ index_root = os.path.join(
+ script.config.distsroot, distroseries.name, component.name)
+
+ source_index_root = os.path.join(index_root, 'source')
+ source_index = RepositoryIndexFile(
+ source_index_root, script.config.temproot, 'Sources')
+ for spp in distroseries.getSourcePackagePublishing(
+ PackagePublishingStatus.PUBLISHED,
+ PackagePublishingPocket.RELEASE, component=component):
+ stanza = spp.getIndexStanza().encode('utf-8') + '\n\n'
+ source_index.write(stanza)
+ source_index.close()
+
+ for arch in distroseries.architectures:
+ package_index_root = os.path.join(
+ index_root, 'binary-%s' % arch.architecturetag)
+ package_index = RepositoryIndexFile(
+ package_index_root, script.config.temproot, 'Packages')
+ for bpp in distroseries.getBinaryPackagePublishing(
+ archtag=arch.architecturetag,
+ pocket=PackagePublishingPocket.RELEASE,
+ component=component):
+ stanza = bpp.getIndexStanza().encode('utf-8') + '\n\n'
+ package_index.write(stanza)
+ package_index.close()
+
+ def makeSeedStructure(self, flavour, series_name, seed_names,
+ seed_inherit=None):
+ """Create a simple seed structure file."""
+ if seed_inherit is None:
+ seed_inherit = {}
+
+ structure_path = os.path.join(
+ self.seeddir, '%s.%s' % (flavour, series_name), 'STRUCTURE')
+ with open_for_writing(structure_path, 'w') as structure:
+ for seed_name in seed_names:
+ if seed_name in seed_inherit:
+ print >>structure, '%s: %s' % (
+ seed_name, ' '.join(seed_inherit[seed_name]))
+ else:
+ print >>structure, '%s:' % seed_name
+
+ def makeSeed(self, flavour, series_name, seed_name, entries,
+ headers=None):
+ """Create a simple seed file."""
+ seed_path = os.path.join(
+ self.seeddir, '%s.%s' % (flavour, series_name), seed_name)
+ with open_for_writing(seed_path, 'w') as seed:
+ if headers is not None:
+ for header in headers:
+ print >>seed, header
+ print >>seed
+ for entry in entries:
+ print >>seed, ' * %s' % entry
+
+ def test_name_is_consistent(self):
+ # Script instances for the same distro get the same name.
+ distro = self.factory.makeDistribution()
+ self.assertEqual(
+ GenerateExtraOverrides(test_args=['-d', distro.name]).name,
+ GenerateExtraOverrides(test_args=['-d', distro.name]).name)
+
+ def test_name_is_unique_for_each_distro(self):
+ # Script instances for different distros get different names.
+ self.assertNotEqual(
+ GenerateExtraOverrides(
+ test_args=['-d', self.factory.makeDistribution().name]).name,
+ GenerateExtraOverrides(
+ test_args=['-d', self.factory.makeDistribution().name]).name)
+
+ def test_requires_distro(self):
+ # The --distribution or -d argument is mandatory.
+ script = GenerateExtraOverrides(test_args=[])
+ script.logger = DevNullLogger()
+ script.txn = FakeTransaction()
+ self.assertRaises(OptionValueError, script.processOptions)
+
+ def test_requires_real_distro(self):
+ # An incorrect distribution name is flagged as an invalid option
+ # value.
+ script = GenerateExtraOverrides(
+ test_args=['-d', self.factory.getUniqueString()])
+ script.logger = DevNullLogger()
+ script.txn = FakeTransaction()
+ self.assertRaises(OptionValueError, script.processOptions)
+
+ def test_looks_up_distro(self):
+ # The script looks up and keeps the distribution named on the
+ # command line.
+ distro = self.makeDistro()
+ self.factory.makeDistroSeries(distro)
+ script = self.makeScript(distro)
+ self.assertEqual(distro, script.distribution)
+
+ def test_prefers_development_distro_series(self):
+ # The script prefers a DEVELOPMENT series for the named
+ # distribution over CURRENT and SUPPORTED series.
+ distro = self.makeDistro()
+ self.factory.makeDistroSeries(distro, status=SeriesStatus.SUPPORTED)
+ self.factory.makeDistroSeries(distro, status=SeriesStatus.CURRENT)
+ development_distroseries = self.factory.makeDistroSeries(
+ distro, status=SeriesStatus.DEVELOPMENT)
+ script = self.makeScript(distro)
+ self.assertEqual(development_distroseries, script.series)
+
+ def test_permits_frozen_distro_series(self):
+ # If there is no DEVELOPMENT series, a FROZEN one will do.
+ distro = self.makeDistro()
+ self.factory.makeDistroSeries(distro, status=SeriesStatus.SUPPORTED)
+ self.factory.makeDistroSeries(distro, status=SeriesStatus.CURRENT)
+ frozen_distroseries = self.factory.makeDistroSeries(
+ distro, status=SeriesStatus.FROZEN)
+ script = self.makeScript(distro)
+ self.assertEqual(frozen_distroseries, script.series)
+
+ def test_requires_development_frozen_distro_series(self):
+ # If there is no DEVELOPMENT or FROZEN series, the script fails.
+ distro = self.makeDistro()
+ self.factory.makeDistroSeries(distro, status=SeriesStatus.SUPPORTED)
+ self.factory.makeDistroSeries(distro, status=SeriesStatus.CURRENT)
+ script = self.makeScript(distro, run_setup=False)
+ self.assertRaises(LaunchpadScriptFailure, script.processOptions)
+
+ def test_components_exclude_partner(self):
+ # If a 'partner' component exists, it is excluded.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distro)
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component="main")
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component="partner")
+ script = self.makeScript(distro)
+ self.assertEqual(["main"], script.components)
+
+ def test_output_path_in_germinateroot(self):
+ # Output files are written to the correct locations under
+ # germinateroot.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distro)
+ script = self.makeScript(distro)
+ flavour = self.factory.getUniqueString()
+ arch = self.factory.getUniqueString()
+ base = self.factory.getUniqueString()
+ output = script.outputPath(flavour, distroseries.name, arch, base)
+ self.assertEqual(
+ '%s/%s_%s_%s_%s' % (
+ script.config.germinateroot, base, flavour, distroseries.name,
+ arch),
+ output)
+
+ def runGerminate(self, script, series_name, arch, flavours):
+ """Helper to call script.runGerminate and return overrides."""
+ structures = script.makeSeedStructures(
+ series_name, flavours, seed_bases=['file://%s' % self.seeddir])
+
+ override_fd, override_path = tempfile.mkstemp()
+ with os.fdopen(override_fd, 'w') as override_file:
+ script.runGerminate(
+ override_file, series_name, arch, flavours, structures)
+ return file_contents(override_path).splitlines()
+
+ def test_germinate_output(self):
+ # A single call to runGerminate produces output for all flavours on
+ # one architecture.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch = das.architecturetag
+ one = self.makePackage(component, [das])
+ two = self.makePackage(component, [das])
+ script = self.makeScript(distro)
+ self.makeIndexFiles(script, distroseries)
+
+ flavour_one = self.factory.getUniqueString()
+ flavour_two = self.factory.getUniqueString()
+ seed = self.factory.getUniqueString()
+ self.makeSeedStructure(flavour_one, series_name, [seed])
+ self.makeSeed(flavour_one, series_name, seed, [one.name])
+ self.makeSeedStructure(flavour_two, series_name, [seed])
+ self.makeSeed(flavour_two, series_name, seed, [two.name])
+
+ overrides = self.runGerminate(
+ script, series_name, arch, [flavour_one, flavour_two])
+ self.assertEqual([], overrides)
+
+ seed_dir_one = os.path.join(
+ self.seeddir, '%s.%s' % (flavour_one, series_name))
+ self.assertFilesEqual(
+ os.path.join(seed_dir_one, 'STRUCTURE'),
+ script.outputPath(flavour_one, series_name, arch, 'structure'))
+ self.assertTrue(file_exists(script.outputPath(
+ flavour_one, series_name, arch, 'all')))
+ self.assertTrue(file_exists(script.outputPath(
+ flavour_one, series_name, arch, 'all.sources')))
+ self.assertTrue(file_exists(script.outputPath(
+ flavour_one, series_name, arch, seed)))
+
+ seed_dir_two = os.path.join(
+ self.seeddir, '%s.%s' % (flavour_two, series_name))
+ self.assertFilesEqual(
+ os.path.join(seed_dir_two, 'STRUCTURE'),
+ script.outputPath(flavour_two, series_name, arch, 'structure'))
+ self.assertTrue(file_exists(script.outputPath(
+ flavour_two, series_name, arch, 'all')))
+ self.assertTrue(file_exists(script.outputPath(
+ flavour_two, series_name, arch, 'all.sources')))
+ self.assertTrue(file_exists(script.outputPath(
+ flavour_two, series_name, arch, seed)))
+
+ def test_germinate_output_task(self):
+ # runGerminate produces Task extra overrides.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch = das.architecturetag
+ one = self.makePackage(component, [das])
+ two = self.makePackage(component, [das], depends=one.name)
+ three = self.makePackage(component, [das])
+ self.makePackage(component, [das])
+ script = self.makeScript(distro)
+ self.makeIndexFiles(script, distroseries)
+
+ flavour = self.factory.getUniqueString()
+ seed_one = self.factory.getUniqueString()
+ seed_two = self.factory.getUniqueString()
+ self.makeSeedStructure(flavour, series_name, [seed_one, seed_two])
+ self.makeSeed(
+ flavour, series_name, seed_one, [two.name],
+ headers=['Task-Description: one'])
+ self.makeSeed(
+ flavour, series_name, seed_two, [three.name],
+ headers=['Task-Description: two'])
+
+ overrides = self.runGerminate(script, series_name, arch, [flavour])
+ expected_overrides = [
+ '%s/%s Task %s' % (one.name, arch, seed_one),
+ '%s/%s Task %s' % (two.name, arch, seed_one),
+ '%s/%s Task %s' % (three.name, arch, seed_two),
+ ]
+ self.assertContentEqual(expected_overrides, overrides)
+
+ def test_germinate_output_task_name(self):
+ # The Task-Name field is honoured.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch = das.architecturetag
+ package = self.makePackage(component, [das])
+ script = self.makeScript(distro)
+ self.makeIndexFiles(script, distroseries)
+
+ flavour = self.factory.getUniqueString()
+ seed_one = self.factory.getUniqueString()
+ task_one = self.factory.getUniqueString()
+ self.makeSeedStructure(flavour, series_name, [seed_one])
+ self.makeSeed(
+ flavour, series_name, seed_one, [package.name],
+ headers=['Task-Name: %s' % task_one])
+
+ overrides = self.runGerminate(script, series_name, arch, [flavour])
+ self.assertContentEqual(
+ ['%s/%s Task %s' % (package.name, arch, task_one)], overrides)
+
+ def test_germinate_output_task_per_derivative(self):
+ # The Task-Per-Derivative field is honoured.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch = das.architecturetag
+ package = self.makePackage(component, [das])
+ script = self.makeScript(distro)
+ self.makeIndexFiles(script, distroseries)
+
+ flavour_one = self.factory.getUniqueString()
+ flavour_two = self.factory.getUniqueString()
+ seed_one = self.factory.getUniqueString()
+ seed_two = self.factory.getUniqueString()
+ self.makeSeedStructure(flavour_one, series_name, [seed_one, seed_two])
+ self.makeSeed(
+ flavour_one, series_name, seed_one, [package.name],
+ headers=['Task-Description: one'])
+ self.makeSeed(
+ flavour_one, series_name, seed_two, [package.name],
+ headers=['Task-Per-Derivative: 1'])
+ self.makeSeedStructure(flavour_two, series_name, [seed_one, seed_two])
+ self.makeSeed(
+ flavour_two, series_name, seed_one, [package.name],
+ headers=['Task-Description: one'])
+ self.makeSeed(
+ flavour_two, series_name, seed_two, [package.name],
+ headers=['Task-Per-Derivative: 1'])
+
+ overrides = self.runGerminate(
+ script, series_name, arch, [flavour_one, flavour_two])
+ # seed_one is not per-derivative, so it is honoured only for
+ # flavour_one and has a global name. seed_two is per-derivative, so
+ # it is honoured for both flavours and has the flavour name
+ # prefixed.
+ expected_overrides = [
+ '%s/%s Task %s' % (package.name, arch, seed_one),
+ '%s/%s Task %s-%s' % (
+ package.name, arch, flavour_one, seed_two),
+ '%s/%s Task %s-%s' % (
+ package.name, arch, flavour_two, seed_two),
+ ]
+ self.assertContentEqual(expected_overrides, overrides)
+
+ def test_germinate_output_task_seeds(self):
+ # The Task-Seeds field is honoured.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch = das.architecturetag
+ one = self.makePackage(component, [das])
+ two = self.makePackage(component, [das])
+ script = self.makeScript(distro)
+ self.makeIndexFiles(script, distroseries)
+
+ flavour = self.factory.getUniqueString()
+ seed_one = self.factory.getUniqueString()
+ seed_two = self.factory.getUniqueString()
+ self.makeSeedStructure(
+ flavour, series_name, [seed_one, seed_two],
+ seed_inherit={seed_two: [seed_one]})
+ self.makeSeed(flavour, series_name, seed_one, [one.name])
+ self.makeSeed(
+ flavour, series_name, seed_two, [two.name],
+ headers=['Task-Seeds: %s' % seed_one])
+
+ overrides = self.runGerminate(script, series_name, arch, [flavour])
+ expected_overrides = [
+ '%s/%s Task %s' % (one.name, arch, seed_two),
+ '%s/%s Task %s' % (two.name, arch, seed_two),
+ ]
+ self.assertContentEqual(expected_overrides, overrides)
+
+ def test_germinate_output_build_essential(self):
+ # runGerminate produces Build-Essential extra overrides.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch = das.architecturetag
+ package = self.makePackage(component, [das])
+ script = self.makeScript(distro)
+ self.makeIndexFiles(script, distroseries)
+
+ flavour = self.factory.getUniqueString()
+ seed = "build-essential"
+ self.makeSeedStructure(flavour, series_name, [seed])
+ self.makeSeed(flavour, series_name, seed, [package.name])
+
+ overrides = self.runGerminate(script, series_name, arch, [flavour])
+ self.assertContentEqual(
+ ['%s/%s Build-Essential yes' % (package.name, arch)], overrides)
+
+ def test_main(self):
+ # If run end-to-end, the script generates override files containing
+ # output for all architectures, and sends germinate's log output to
+ # a file.
+ distro = self.makeDistro()
+ distroseries = self.factory.makeDistroSeries(distribution=distro)
+ series_name = distroseries.name
+ component = self.factory.makeComponent()
+ self.factory.makeComponentSelection(
+ distroseries=distroseries, component=component)
+ das_one = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch_one = das_one.architecturetag
+ das_two = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ arch_two = das_two.architecturetag
+ package = self.makePackage(component, [das_one, das_two])
+ flavour = self.factory.getUniqueString()
+ script = self.makeScript(distro, flavours=[flavour])
+ self.makeIndexFiles(script, distroseries)
+
+ seed = self.factory.getUniqueString()
+ self.makeSeedStructure(flavour, series_name, [seed])
+ self.makeSeed(
+ flavour, series_name, seed, [package.name],
+ headers=['Task-Description: task'])
+
+ script.process(seed_bases=['file://%s' % self.seeddir])
+ override_path = os.path.join(
+ script.config.miscroot,
+ "more-extra.override.%s.main" % series_name)
+ expected_overrides = [
+ '%s/%s Task %s' % (package.name, arch_one, seed),
+ '%s/%s Task %s' % (package.name, arch_two, seed),
+ ]
+ self.assertContentEqual(
+ expected_overrides, file_contents(override_path).splitlines())
+
+ log_file = os.path.join(
+ script.config.germinateroot, 'germinate.output')
+ self.assertIn('Downloading file://', file_contents(log_file))
Follow ups
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Jeroen T. Vermeulen, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Colin Watson, 2011-12-07
-
Re: [Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Colin Watson, 2011-12-06
-
[Merge] lp:~cjwatson/launchpad/refactor-cron-germinate into lp:launchpad
From: Colin Watson, 2011-12-06
