launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #09836
[Merge] lp:~cjwatson/launchpad/remove-queue-tool into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/remove-queue-tool into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #1006173 in Launchpad itself: "Queue tool requires direct DB access"
https://bugs.launchpad.net/launchpad/+bug/1006173
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/remove-queue-tool/+merge/114464
== Summary ==
Complete the migration from the queue script to an API client.
== Proposed fix ==
Once the current contents of devel pass buildbot and QA and are deployed, the queue API client will be feature-complete. We can and should therefore remove the old queue script to reclaim the lines of code spent on the API work (twice over, in fact).
== Implementation details ==
Many of the tests were already exercised elsewhere (notably, overrides are I think already reasonably well exercised via TestPackageUploadWebservice), and some of them were specific to the script and are thus obsolete, but some of them were neither. I moved the tests of DistroSeries:+queue closing private bugs to lp.soyuz.scripts.tests.test_processaccepted (there was already a comment indicating that that needed to happen); and a number of the accept/reject tests were essentially testing the model, so I turned them into proper model tests in lp.soyuz.tests.test_packageupload.
== Tests ==
bin/test -vvct test_packageupload -t test_processaccepted
== Lint ==
Just an existing false positive due to pocketlint not understanding property setters:
./lib/lp/soyuz/model/sourcepackagerelease.py
196: redefinition of function 'copyright' from line 187
--
https://code.launchpad.net/~cjwatson/launchpad/remove-queue-tool/+merge/114464
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/remove-queue-tool into lp:launchpad.
=== removed file 'lib/lp/soyuz/doc/ftpmaster-tools.txt'
--- lib/lp/soyuz/doc/ftpmaster-tools.txt 2011-12-29 05:29:36 +0000
+++ lib/lp/soyuz/doc/ftpmaster-tools.txt 1970-01-01 00:00:00 +0000
@@ -1,71 +0,0 @@
-= FTPMASTER Tools =
-
-This test commits to the test database in subprocesses and so needs to
-force the DatabaseLayer to fully tear down and restore the database
-after this test.
-
- >>> from lp.testing.layers import DatabaseLayer
- >>> DatabaseLayer.force_dirty_database()
-
-Queue Tool is a script designed to handle queue content.
-This test will check its output, since the script itself would
-open a new connection, let's invoke it in dry-run mode.
-
- >>> import subprocess
- >>> import os
- >>> import sys
- >>> from lp.services.config import config
-
-
- >>> script = os.path.join(config.root, "scripts", "ftpmaster-tools",
- ... "queue")
-
-INFO
-
- >>> process = subprocess.Popen([sys.executable, script,
- ... "-s", "breezy-autotest", "info"],
- ... stdout=subprocess.PIPE)
- >>> stdout, stderr = process.communicate()
- >>> process.returncode
- 0
- >>> print stdout
- Initializing connection to queue new
- Running: "info"
- Listing ubuntu/breezy-autotest (NEW) 6/6
- ---------|----|----------------------|----------------------|---------------
- 7 | -- | netapplet-1.0.0.tar. | - | ...
- | * netapplet-1.0.0.tar.gz Format: DDTP_TARBALL
- 6 | -- | netapplet-1.0.0.tar. | - | ...
- | * netapplet-1.0.0.tar.gz Format: DIST_UPGRADER
- 4 | S- | alsa-utils | 1.0.9a-4ubuntu1 | ...
- | * alsa-utils/1.0.9a-4ubuntu1 Component: main Section: base
- 3 | S- | netapplet | 0.99.6-1 | ...
- | * netapplet/0.99.6-1 Component: main Section: web
- 2 | -B | pmount (i386) | 0.1-1 | ...
- | N pmount/0.1-1/i386 Component: main Section: base Priority: IMPORTANT
- 1 | -B | mozilla-firefox (i38 | 0.9 | ...
- | N mozilla-firefox/0.9/i386 Component: main Section: base Priority: EXTRA
- ---------|----|----------------------|----------------------|---------------
- 6/6
- total
-
-
-Check the custom uploads presentation:
-
- >>> process = subprocess.Popen([sys.executable, script, "-Q", "unapproved",
- ... "-s", "breezy-autotest-updates", "info"],
- ... stdout=subprocess.PIPE)
- >>> stdout, stderr = process.communicate()
- >>> process.returncode
- 0
- >>> print stdout
- Initializing connection to queue unapproved
- Running: "info"
- Listing ubuntu/breezy-autotest-updates (UNAPPROVED) 1/1
- ---------|----|----------------------|----------------------|---------------
- 5 | -- | netapplet-1.0.0.tar. | - | ...
- | * netapplet-1.0.0.tar.gz Format: ROSETTA_TRANSLATIONS
- ---------|----|----------------------|----------------------|---------------
- 1/1 total
- <BLANKLINE>
-
=== modified file 'lib/lp/soyuz/model/sourcepackagerelease.py'
--- lib/lp/soyuz/model/sourcepackagerelease.py 2012-07-09 12:32:23 +0000
+++ lib/lp/soyuz/model/sourcepackagerelease.py 2012-07-11 16:34:21 +0000
@@ -66,6 +66,7 @@
from lp.soyuz.interfaces.archive import MAIN_ARCHIVE_PURPOSES
from lp.soyuz.interfaces.binarypackagebuild import IBinaryPackageBuildSet
from lp.soyuz.interfaces.packagediff import PackageDiffAlreadyRequested
+from lp.soyuz.interfaces.queue import QueueInconsistentStateError
from lp.soyuz.interfaces.sourcepackagerelease import ISourcePackageRelease
from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
from lp.soyuz.model.files import SourcePackageReleaseFile
@@ -74,7 +75,6 @@
PackageUpload,
PackageUploadSource,
)
-from lp.soyuz.scripts.queue import QueueActionError
from lp.translations.interfaces.translationimportqueue import (
ITranslationImportQueue,
)
@@ -474,7 +474,7 @@
if new_archive is not None:
self.upload_archive = new_archive
else:
- raise QueueActionError(
+ raise QueueInconsistentStateError(
"New component '%s' requires a non-existent archive.")
if section is not None:
self.section = section
=== removed file 'lib/lp/soyuz/scripts/queue.py'
--- lib/lp/soyuz/scripts/queue.py 2012-07-03 16:04:54 +0000
+++ lib/lp/soyuz/scripts/queue.py 1970-01-01 00:00:00 +0000
@@ -1,737 +0,0 @@
-# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-# pylint: disable-msg=W0231
-"""Ftpmaster queue tool libraries."""
-
-# XXX StuartBishop 2007-01-31:
-# This should be renamed to ftpmasterqueue.py or just ftpmaster.py
-# as Launchpad contains lots of queues.
-
-__metaclass__ = type
-__all__ = [
- 'CommandRunner',
- 'CommandRunnerError',
- 'QueueActionError',
- 'name_queue_map',
- ]
-
-from datetime import datetime
-import errno
-import hashlib
-
-import pytz
-from zope.component import getUtility
-
-from lp.app.browser.tales import DurationFormatterAPI
-from lp.app.errors import NotFoundError
-from lp.services.config import config
-from lp.services.librarian.utils import filechunks
-from lp.services.propertycache import cachedproperty
-from lp.soyuz.enums import PackageUploadStatus
-from lp.soyuz.interfaces.component import IComponentSet
-from lp.soyuz.interfaces.queue import (
- IPackageUploadSet,
- QueueInconsistentStateError,
- )
-from lp.soyuz.interfaces.section import ISectionSet
-
-
-name_queue_map = {
- "new": PackageUploadStatus.NEW,
- "unapproved": PackageUploadStatus.UNAPPROVED,
- "accepted": PackageUploadStatus.ACCEPTED,
- "done": PackageUploadStatus.DONE,
- "rejected": PackageUploadStatus.REJECTED,
- }
-
-#XXX cprov 2006-09-19: We need to use template engine instead of harcoded
-# format variables.
-HEAD = "-" * 9 + "|----|" + "-" * 22 + "|" + "-" * 22 + "|" + "-" * 15
-FOOT_MARGIN = " " * (9 + 6 + 1 + 22 + 1 + 22 + 2)
-RULE = "-" * (12 + 9 + 6 + 1 + 22 + 1 + 22 + 2)
-
-FILTERMSG = """
- Omit the filter for all records.
- Filter string consists of a queue ID or a pair <name>[/<version>]:
-
- 28
- apt
- apt/1
-
- Use '-e' command line argument for exact matches:
-
- -e apt
- -e apt/1.0-1
-"""
-
-
-class QueueActionError(Exception):
- """Identify Errors occurred within QueueAction class and its children."""
-
-
-class QueueAction:
- """Queue Action base class.
-
- Implements a bunch of common/useful method designed to provide easy
- PackageUpload handling.
- """
-
- def __init__(self, distribution_name, suite_name, queue, terms,
- component_name, section_name, priority_name,
- display, no_mail=True, exact_match=False, log=None):
- """Initializes passed variables. """
- self.terms = terms
- # Some actions have addtional commands at the start of the terms
- # so allow them to state that here by specifiying the start index.
- self.terms_start_index = 0
- self.component_name = component_name
- self.section_name = section_name
- self.priority_name = priority_name
- self.exact_match = exact_match
- self.queue = queue
- self.no_mail = no_mail
- self.distribution_name = distribution_name
- self.suite_name = suite_name
- self.default_sender = "%s <%s>" % (
- config.uploader.default_sender_name,
- config.uploader.default_sender_address)
- self.default_recipient = "%s <%s>" % (
- config.uploader.default_recipient_name,
- config.uploader.default_recipient_address)
- self.display = display
- self.log = log
-
- @cachedproperty
- def size(self):
- """Return the size of the queue in question."""
- return getUtility(IPackageUploadSet).count(
- status=self.queue, distroseries=self.distroseries,
- pocket=self.pocket)
-
- def setDefaultContext(self):
- """Set default distribuiton, distroseries."""
- # if not found defaults to 'ubuntu'
-
- # Avoid circular imports.
- from lp.registry.interfaces.distribution import IDistributionSet
- from lp.registry.interfaces.pocket import PackagePublishingPocket
-
- distroset = getUtility(IDistributionSet)
- try:
- self.distribution = distroset[self.distribution_name]
- except NotFoundError:
- self.distribution = distroset['ubuntu']
-
- if self.suite_name:
- # defaults to distro.currentseries if passed distroseries is
- # misapplied or not found.
- try:
- self.distroseries, self.pocket = (
- self.distribution.getDistroSeriesAndPocket(
- self.suite_name))
- except NotFoundError:
- raise QueueActionError('Context not found: "%s/%s"'
- % (self.distribution.name,
- self.suite_name))
- else:
- self.distroseries = self.distribution.currentseries
- self.pocket = PackagePublishingPocket.RELEASE
-
- def initialize(self):
- """Builds a list of affected records based on the filter argument."""
- self.setDefaultContext()
-
- self.package_names = []
- self.items = []
- self.items_size = 0
-
- # Will be set to true if the command line specified package IDs.
- # This is required because package_names is expanded into IDs so we
- # need another way of knowing whether the user typed them.
- self.explicit_ids_specified = False
-
- terms = self.terms[self.terms_start_index:]
- if len(terms) == 0:
- # If no argument is passed, present all available results in
- # the selected queue.
- terms.append('')
-
- for term in terms:
- # refuse old-style '*' argument since we do not support
- # wildcards yet.
- if term == '*':
- self.displayUsage(FILTERMSG)
-
- if term.isdigit():
- # retrieve PackageUpload item by id
- try:
- item = getUtility(IPackageUploadSet).get(int(term))
- except NotFoundError as info:
- raise QueueActionError('Queue Item not found: %s' % info)
-
- if item.status != self.queue:
- raise QueueActionError(
- 'Item %s is in queue %s' % (
- item.id, item.status.name))
-
- if (item.distroseries != self.distroseries or
- item.pocket != self.pocket):
- raise QueueActionError(
- 'Item %s is in %s/%s-%s not in %s/%s-%s'
- % (item.id, item.distroseries.distribution.name,
- item.distroseries.name, item.pocket.name,
- self.distroseries.distribution.name,
- self.distroseries.name, self.pocket.name))
-
- if item not in self.items:
- self.items.append(item)
- self.explicit_ids_specified = True
- else:
- # retrieve PackageUpload item by name/version key
- version = None
- if '/' in term:
- term, version = term.strip().split('/')
-
- # Expand SQLObject results.
- queue_items = self.distroseries.getPackageUploads(
- status=self.queue, name=term, version=version,
- exact_match=self.exact_match, pocket=self.pocket)
- for item in queue_items:
- if item not in self.items:
- self.items.append(item)
- self.package_names.append(term)
-
- self.items_size = len(self.items)
-
- def run(self):
- """Place holder for command action."""
- raise NotImplementedError('No action implemented.')
-
- def displayTitle(self, action):
- """Common title/summary presentation method."""
- self.display("%s %s/%s (%s) %s/%s" % (
- action, self.distribution.name, self.suite_name,
- self.queue.name, self.items_size, self.size))
-
- def displayHead(self):
- """Table head presentation method."""
- self.display(HEAD)
-
- def displayBottom(self):
- """Displays the table bottom and a small statistic information."""
- self.display(
- FOOT_MARGIN + "%d/%d total" % (self.items_size, self.size))
-
- def displayRule(self):
- """Displays a rule line. """
- self.display(RULE)
-
- def displayUsage(self, extended_info=None):
- """Display the class docstring as usage message.
-
- Raise QueueActionError with optional extended_info argument
- """
- self.display(self.__doc__)
- raise QueueActionError(extended_info)
-
- def _makeTag(self, queue_item):
- """Compose an upload type tag for `queue_item`.
-
- A source upload without binaries is tagged as "S-".
- A binary upload without source is tagged as "-B."
- An upload with both source and binaries is tagged as "SB".
- An upload with a package copy job is tagged as "X-".
- """
- # XXX cprov 2006-07-31: source_tag and build_tag ('S' & 'B')
- # are necessary simply to keep the format legaxy.
- # We may discuss a more reasonable output format later
- # and avoid extra boring code. The IDRQ.displayname should
- # do should be enough.
- if queue_item.package_copy_job is not None:
- return "X-"
-
- source_tag = {
- True: 'S',
- False: '-',
- }
- binary_tag = {
- True: 'B',
- False: '-',
- }
- return (
- source_tag[queue_item.contains_source] +
- binary_tag[queue_item.contains_build])
-
- def displayItem(self, queue_item):
- """Display one line summary of the queue item provided."""
- tag = self._makeTag(queue_item)
- displayname = queue_item.displayname
- version = queue_item.displayversion
- age = DurationFormatterAPI(
- datetime.now(pytz.timezone('UTC')) -
- queue_item.date_created).approximateduration()
-
- if queue_item.contains_build:
- displayname = "%s (%s)" % (queue_item.displayname,
- queue_item.displayarchs)
-
- self.display("%8d | %s | %s | %s | %s" %
- (queue_item.id, tag, displayname.ljust(20)[:20],
- version.ljust(20)[:20], age))
-
- def displayInfo(self, queue_item, only=None):
- """Displays additional information about the provided queue item.
-
- Optionally pass a binarypackagename via 'only' argument to display
- only exact matches within the selected build queue items.
- """
- if queue_item.package_copy_job or queue_item.sources:
- self.display(
- "\t | * %s/%s Component: %s Section: %s" % (
- queue_item.package_name,
- queue_item.package_version,
- queue_item.component_name,
- queue_item.section_name,
- ))
-
- for queue_build in queue_item.builds:
- for bpr in queue_build.build.binarypackages:
- if only and only != bpr.name:
- continue
- if bpr.is_new:
- status_flag = "N"
- else:
- status_flag = "*"
- self.display(
- "\t | %s %s/%s/%s Component: %s Section: %s Priority: %s"
- % (status_flag, bpr.name, bpr.version,
- bpr.build.distro_arch_series.architecturetag,
- bpr.component.name, bpr.section.name,
- bpr.priority.name))
-
- for queue_custom in queue_item.customfiles:
- self.display("\t | * %s Format: %s"
- % (queue_custom.libraryfilealias.filename,
- queue_custom.customformat.name))
-
-
-class QueueActionHelp(QueueAction):
- """Present provided actions summary"""
-
- def __init__(self, **kargs):
- self.kargs = kargs
- self.kargs['no_mail'] = True
- self.actions = kargs['terms']
- self.display = kargs['display']
-
- def initialize(self):
- """Mock initialization """
- pass
-
- def run(self):
- """Present the actions description summary"""
- # present summary for specific or all actions
- if not self.actions:
- actions_help = queue_actions.items()
- not_available_actions = []
- else:
- actions_help = [
- (action, provider)
- for action, provider in queue_actions.items()
- if action in self.actions]
- not_available_actions = [
- action for action in self.actions
- if action not in queue_actions.keys()]
- # present not available requested action if any.
- if not_available_actions:
- self.display(
- "Not available action(s): %s" %
- ", ".join(not_available_actions))
-
- # extract summary from docstring of specified available actions
- for action, wrapper in actions_help:
- if action is 'help':
- continue
- wobj = wrapper(**self.kargs)
- summary = wobj.__doc__.splitlines()[0]
- self.display('\t%s : %s ' % (action, summary))
-
-
-class QueueActionReport(QueueAction):
- """Present a report about the size of available queues"""
-
- def initialize(self):
- """Mock initialization """
- self.setDefaultContext()
-
- def run(self):
- """Display the queues size."""
- self.display("Report for %s/%s" % (self.distribution.name,
- self.distroseries.name))
-
- for queue in name_queue_map.values():
- size = getUtility(IPackageUploadSet).count(
- status=queue, distroseries=self.distroseries,
- pocket=self.pocket)
- self.display("\t%s -> %s entries" % (queue.name, size))
-
-
-class QueueActionInfo(QueueAction):
- """Present the Queue item including its contents.
-
- Presents the contents of the selected upload(s).
-
- queue info <filter>
- """
-
- def run(self):
- """Present the filtered queue ordered by date."""
- self.displayTitle('Listing')
- self.displayHead()
- for queue_item in self.items:
- self.displayItem(queue_item)
- self.displayInfo(queue_item)
- self.displayHead()
- self.displayBottom()
-
-
-class QueueActionFetch(QueueAction):
- """Fetch the contents of a queue item.
-
- Download the contents of the selected upload(s).
-
- queue fetch <filter>
- """
-
- def run(self):
- self.displayTitle('Fetching')
- self.displayRule()
- for queue_item in self.items:
- file_list = []
- if queue_item.changesfile is not None:
- file_list.append(queue_item.changesfile)
-
- for source in queue_item.sources:
- for spr_file in source.sourcepackagerelease.files:
- file_list.append(spr_file.libraryfile)
-
- for build in queue_item.builds:
- for bpr in build.build.binarypackages:
- for bpr_file in bpr.files:
- file_list.append(bpr_file.libraryfile)
-
- for custom in queue_item.customfiles:
- file_list.append(custom.libraryfilealias)
-
- for libfile in file_list:
- self.display("Constructing %s" % libfile.filename)
- # do not overwrite files on disk (bug # 62976)
- try:
- existing_file = open(libfile.filename, "r")
- except IOError as e:
- if not e.errno == errno.ENOENT:
- raise
- # File does not already exist, so read file from librarian
- # and write to disk.
- libfile.open()
- out_file = open(libfile.filename, "w")
- for chunk in filechunks(libfile):
- out_file.write(chunk)
- out_file.close()
- libfile.close()
- else:
- # Check sha against existing file (bug #67014)
- existing_sha = hashlib.sha1()
- for chunk in filechunks(existing_file):
- existing_sha.update(chunk)
- existing_file.close()
-
- # bail out if the sha1 differs
- if libfile.content.sha1 != existing_sha.hexdigest():
- raise CommandRunnerError("%s already present on disk "
- "and differs from new file"
- % libfile.filename)
- else:
- self.display("%s already on disk and checksum "
- "matches, skipping.")
-
- self.displayRule()
- self.displayBottom()
-
-
-class QueueActionReject(QueueAction):
- """Reject the contents of a queue item.
-
- Move the selected upload(s) to the REJECTED queue.
-
- queue reject <filter>
- """
-
- def run(self):
- """Perform Reject action."""
- self.displayTitle('Rejecting')
- self.displayRule()
- for queue_item in self.items:
- self.display('Rejecting %s' % queue_item.displayname)
- try:
- queue_item.rejectFromQueue(
- logger=self.log, dry_run=self.no_mail)
- except QueueInconsistentStateError as info:
- self.display('** %s could not be rejected due %s'
- % (queue_item.displayname, info))
-
- self.displayRule()
- self.displayBottom()
-
-
-class QueueActionAccept(QueueAction):
- """Accept the contents of a queue item.
-
- Move the selected upload(s) to the ACCEPTED queue.
-
- queue accept <filter>
- """
-
- def run(self):
- """Perform Accept action."""
- self.displayTitle('Accepting')
- self.displayRule()
- for queue_item in self.items:
- self.display('Accepting %s' % queue_item.displayname)
- try:
- queue_item.acceptFromQueue(
- logger=self.log, dry_run=self.no_mail)
- except QueueInconsistentStateError as info:
- self.display('** %s could not be accepted due to %s'
- % (queue_item.displayname, info))
- continue
-
- self.displayRule()
- self.displayBottom()
-
-
-class QueueActionOverride(QueueAction):
- """Override information in a queue item content.
-
- queue override [-c|--component] [-x|--section] [-p|--priority]
- <override_stanza> <filter>
-
- Where override_stanza is one of:
- source
- binary
-
- In each case, when you want to set an override supply the relevant option.
-
- So, to set a binary to have section 'editors' but leave the
- component and priority alone, do:
-
- queue override -x editors binary <filter>
-
- Binaries can only be overridden by passing a name filter, so it will
- only override the binary package which matches the filter.
-
- Or, to set a source's section to editors, do:
-
- queue override -x editors source <filter>
- """
- supported_override_stanzas = ['source', 'binary']
-
- def __init__(self, distribution_name, suite_name, queue, terms,
- component_name, section_name, priority_name,
- display, no_mail=True, exact_match=False, log=None):
- """Constructor for QueueActionOverride."""
-
- # This exists so that self.terms_start_index can be set as this action
- # class has a command at the start of the terms.
- # Our first term is "binary" or "source" to specify the type of
- # over-ride.
- QueueAction.__init__(self, distribution_name, suite_name, queue,
- terms, component_name, section_name,
- priority_name, display, no_mail=True,
- exact_match=False, log=log)
- self.terms_start_index = 1
- self.overrides_performed = 0
-
- def run(self):
- """Perform Override action."""
- self.displayTitle('Overriding')
- self.displayRule()
-
- # "terms" is the list of arguments starting at the override stanza
- # ("source" or "binary").
- try:
- override_stanza = self.terms[0]
- except IndexError:
- self.displayUsage('Missing override_stanza.')
- return
-
- if override_stanza not in self.supported_override_stanzas:
- self.displayUsage('Not supported override_stanza: %s'
- % override_stanza)
- return
-
- return getattr(self, '_override_' + override_stanza)()
-
- def _override_source(self):
- """Overrides sourcepackagereleases selected.
-
- It doesn't check Component/Section Selection, this is a task
- for queue state-machine.
- """
- component = None
- section = None
- try:
- if self.component_name:
- component = getUtility(IComponentSet)[self.component_name]
- if self.section_name:
- section = getUtility(ISectionSet)[self.section_name]
- except NotFoundError as info:
- raise QueueActionError('Not Found: %s' % info)
-
- for queue_item in self.items:
- # We delegate to the queue_item itself to override any/all
- # of its sources.
- if queue_item.contains_source or queue_item.package_copy_job:
- if queue_item.sourcepackagerelease:
- old_component = queue_item.sourcepackagerelease.component
- else:
- old_component = getUtility(IComponentSet)[
- queue_item.package_copy_job.component_name]
- queue_item.overrideSource(
- component, section, [
- component, old_component])
- self.overrides_performed += 1
- self.displayInfo(queue_item)
-
- def _override_binary(self):
- """Overrides binarypackagereleases selected"""
- from lp.soyuz.interfaces.publishing import name_priority_map
- if self.explicit_ids_specified:
- self.displayUsage('Cannot Override BinaryPackage retrieved by ID')
-
- component = None
- section = None
- priority = None
- try:
- if self.component_name:
- component = getUtility(IComponentSet)[self.component_name]
- if self.section_name:
- section = getUtility(ISectionSet)[self.section_name]
- if self.priority_name:
- priority = name_priority_map[self.priority_name]
- except (NotFoundError, KeyError) as info:
- raise QueueActionError('Not Found: %s' % info)
-
- overridden = []
- for queue_item in self.items:
- for build in queue_item.builds:
- # Different than PackageUploadSources
- # PackageUploadBuild points to a Build, that can,
- # and usually does, point to multiple BinaryPackageReleases.
- # So we need to carefully select the requested package to be
- # overridden
- for binary in build.build.binarypackages:
- if binary.name in self.package_names:
- overridden.append(binary.name)
- self.display("Overriding %s_%s (%s/%s/%s)"
- % (binary.name, binary.version,
- binary.component.name,
- binary.section.name,
- binary.priority.name))
- binary.override(component=component, section=section,
- priority=priority)
- self.overrides_performed += 1
- self.displayInfo(queue_item, only=binary.name)
- # See if the new component requires a new archive on the
- # build:
- if component:
- distroarchseries = build.build.distro_arch_series
- distribution = distroarchseries.distroseries.distribution
- new_archive = distribution.getArchiveByComponent(
- self.component_name)
- if (new_archive != build.build.archive):
- raise QueueActionError(
- "Overriding component to '%s' failed because it "
- "would require a new archive."
- % self.component_name)
-
- not_overridden = set(self.package_names) - set(overridden)
- if len(not_overridden) > 0:
- self.displayUsage('No matches for %s' % ",".join(not_overridden))
-
-
-queue_actions = {
- 'help': QueueActionHelp,
- 'info': QueueActionInfo,
- 'fetch': QueueActionFetch,
- 'accept': QueueActionAccept,
- 'reject': QueueActionReject,
- 'override': QueueActionOverride,
- 'report': QueueActionReport,
- }
-
-
-def default_display(text):
- """Unified presentation method."""
- print text
-
-
-class CommandRunnerError(Exception):
- """Command Runner Failure"""
-
-
-class CommandRunner:
- """A wrapper for queue_action classes."""
-
- def __init__(self, queue, distribution_name, suite_name,
- no_mail, component_name, section_name, priority_name,
- display=default_display, log=None):
- self.queue = queue
- self.distribution_name = distribution_name
- self.suite_name = suite_name
- self.no_mail = no_mail
- self.component_name = component_name
- self.section_name = section_name
- self.priority_name = priority_name
- self.display = display
- self.log = log
-
- def execute(self, terms, exact_match=False):
- """Execute a single queue action."""
- self.display('Running: "%s"' % " ".join(terms))
-
- # check syntax, abort process if anything gets wrong
- try:
- action = terms[0]
- arguments = [unicode(term) for term in terms[1:]]
- except IndexError:
- raise CommandRunnerError('Invalid sentence, use help.')
-
- # check action availability,
- try:
- queue_action_class = queue_actions[action]
- except KeyError:
- raise CommandRunnerError('Unknown Action: %s' % action)
-
- # perform the required action on queue.
- try:
- # be sure to send every args via kargs
- queue_action = queue_action_class(
- distribution_name=self.distribution_name,
- suite_name=self.suite_name,
- queue=self.queue,
- no_mail=self.no_mail,
- display=self.display,
- terms=arguments,
- component_name=self.component_name,
- section_name=self.section_name,
- priority_name=self.priority_name,
- exact_match=exact_match,
- log=self.log)
- queue_action.initialize()
- queue_action.run()
- except QueueActionError as info:
- raise CommandRunnerError(info)
-
- return queue_action
=== modified file 'lib/lp/soyuz/scripts/tests/test_processaccepted.py'
--- lib/lp/soyuz/scripts/tests/test_processaccepted.py 2012-01-01 02:58:52 +0000
+++ lib/lp/soyuz/scripts/tests/test_processaccepted.py 2012-07-11 16:34:21 +0000
@@ -1,20 +1,30 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
__metaclass__ = type
+from StringIO import StringIO
from textwrap import dedent
+from zope.security.interfaces import ForbiddenAttribute
from zope.security.proxy import removeSecurityProxy
from lp.bugs.interfaces.bugtask import BugTaskStatus
+from lp.registry.enums import InformationType
from lp.registry.interfaces.pocket import PackagePublishingPocket
from lp.soyuz.scripts.processaccepted import (
close_bugs_for_sourcepackagerelease,
close_bugs_for_sourcepublication,
)
-from lp.testing import TestCaseWithFactory
-from lp.testing.layers import LaunchpadZopelessLayer
+from lp.testing import (
+ celebrity_logged_in,
+ person_logged_in,
+ TestCaseWithFactory,
+ )
+from lp.testing.layers import (
+ DatabaseFunctionalLayer,
+ LaunchpadZopelessLayer,
+ )
class TestClosingBugs(TestCaseWithFactory):
@@ -24,7 +34,6 @@
start a unification in a single file and those other tests need
migrating here.
See also:
- * lp/soyuz/scripts/tests/test_queue.py
* lib/lp/soyuz/doc/closing-bugs-from-changelogs.txt
* lib/lp/archiveuploader/tests/nascentupload-closing-bugs.txt
"""
@@ -127,3 +136,34 @@
for bug, bugtask in bugs:
self.assertEqual(BugTaskStatus.FIXRELEASED, bugtask.status)
+
+
+class TestClosingPrivateBugs(TestCaseWithFactory):
+ # The distroseries +queue page can close private bugs when accepting
+ # packages.
+
+ layer = DatabaseFunctionalLayer
+
+ def test_close_bugs_for_sourcepackagerelease_with_private_bug(self):
+ """close_bugs_for_sourcepackagerelease works with private bugs."""
+ changes_file_template = "Format: 1.7\nLaunchpad-bugs-fixed: %s\n"
+ # changelog_entry is required for an assertion inside the function
+ # we're testing.
+ spr = self.factory.makeSourcePackageRelease(changelog_entry="blah")
+ archive_admin = self.factory.makePerson()
+ bug = self.factory.makeBug(
+ sourcepackagename=spr.sourcepackagename,
+ distribution=spr.upload_distroseries.distribution,
+ information_type=InformationType.USERDATA)
+ changes = StringIO(changes_file_template % bug.id)
+
+ with person_logged_in(archive_admin):
+ # The archive admin user can't normally see this bug.
+ self.assertRaises(ForbiddenAttribute, bug, 'status')
+ # But the bug closure should work.
+ close_bugs_for_sourcepackagerelease(spr, changes)
+
+ # Verify it was closed.
+ with celebrity_logged_in("admin"):
+ self.assertEqual(
+ bug.default_bugtask.status, BugTaskStatus.FIXRELEASED)
=== removed file 'lib/lp/soyuz/scripts/tests/test_queue.py'
--- lib/lp/soyuz/scripts/tests/test_queue.py 2012-05-31 05:53:09 +0000
+++ lib/lp/soyuz/scripts/tests/test_queue.py 1970-01-01 00:00:00 +0000
@@ -1,1262 +0,0 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""queue tool base class tests."""
-
-__metaclass__ = type
-
-import hashlib
-import os
-import shutil
-from StringIO import StringIO
-import tempfile
-from unittest import TestCase
-
-from testtools.matchers import StartsWith
-from zope.component import getUtility
-from zope.security.interfaces import ForbiddenAttribute
-from zope.security.proxy import removeSecurityProxy
-
-from lp.archiveuploader.nascentupload import NascentUpload
-from lp.archiveuploader.tests import (
- datadir,
- getPolicy,
- insertFakeChangesFileForAllPackageUploads,
- )
-from lp.bugs.interfaces.bug import IBugSet
-from lp.bugs.interfaces.bugtask import (
- BugTaskStatus,
- IBugTaskSet,
- )
-from lp.registry.enums import InformationType
-from lp.registry.interfaces.distribution import IDistributionSet
-from lp.registry.interfaces.person import IPersonSet
-from lp.registry.interfaces.pocket import PackagePublishingPocket
-from lp.registry.interfaces.series import SeriesStatus
-from lp.services.config import config
-from lp.services.database.lpstorm import IStore
-from lp.services.librarian.interfaces import ILibraryFileAliasSet
-from lp.services.librarian.model import LibraryFileAlias
-from lp.services.librarian.utils import filechunks
-from lp.services.librarianserver.testing.server import fillLibrarianFile
-from lp.services.log.logger import DevNullLogger
-from lp.services.mail import stub
-from lp.soyuz.enums import (
- ArchivePurpose,
- PackagePublishingStatus,
- PackageUploadStatus,
- )
-from lp.soyuz.interfaces.archive import IArchiveSet
-from lp.soyuz.interfaces.queue import IPackageUploadSet
-from lp.soyuz.model.queue import PackageUploadBuild
-from lp.soyuz.scripts.processaccepted import (
- close_bugs_for_sourcepackagerelease,
- )
-from lp.soyuz.scripts.queue import (
- CommandRunner,
- CommandRunnerError,
- name_queue_map,
- QueueAction,
- QueueActionOverride,
- )
-from lp.testing import (
- celebrity_logged_in,
- person_logged_in,
- TestCaseWithFactory,
- )
-from lp.testing.dbuser import (
- dbuser,
- lp_dbuser,
- switch_dbuser,
- )
-from lp.testing.fakemethod import FakeMethod
-from lp.testing.layers import (
- DatabaseFunctionalLayer,
- LaunchpadZopelessLayer,
- LibrarianLayer,
- )
-
-
-class TestQueueBase:
- """Base methods for queue tool test classes."""
-
- def setUp(self):
- # Switch database user and set isolation level to READ COMMIITTED
- # to avoid SERIALIZATION exceptions with the Librarian.
- switch_dbuser(self.dbuser)
-
- def _test_display(self, text):
- """Store output from queue tool for inspection."""
- self.test_output.append(text)
-
- def execute_command(self, argument, queue_name='new', no_mail=True,
- distribution_name='ubuntu', component_name=None,
- section_name=None, priority_name=None,
- suite_name='breezy-autotest', quiet=True):
- """Helper method to execute a queue command.
-
- Initialize output buffer and execute a command according
- given argument.
-
- Return the used QueueAction instance.
- """
- self.test_output = []
- queue = name_queue_map[queue_name]
- runner = CommandRunner(
- queue, distribution_name, suite_name, no_mail,
- component_name, section_name, priority_name,
- display=self._test_display)
-
- return runner.execute(argument.split())
-
- def assertEmail(self, expected_to_addrs):
- """Pop an email from the stub queue and check its recipients."""
- from_addr, to_addrs, raw_msg = stub.test_emails.pop()
- self.assertEqual(to_addrs, expected_to_addrs)
-
-
-class TestQueueTool(TestQueueBase, TestCase):
- layer = LaunchpadZopelessLayer
- dbuser = config.uploadqueue.dbuser
-
- def setUp(self):
- """Create contents in disk for librarian sampledata."""
- # Packageupload.notify() needs real changes file data to send
- # email, so this nice simple "ed" changes file will do. It's
- # the /wrong/ changes file for the package in the upload queue,
- # but that doesn't matter as only email addresses are parsed out
- # of it.
- insertFakeChangesFileForAllPackageUploads()
- fake_chroot = LibraryFileAlias.get(1)
-
- switch_dbuser("testadmin")
-
- ubuntu = getUtility(IDistributionSet)['ubuntu']
- breezy_autotest = ubuntu.getSeries('breezy-autotest')
- breezy_autotest['i386'].addOrUpdateChroot(fake_chroot)
-
- switch_dbuser('launchpad')
-
- TestQueueBase.setUp(self)
-
- def tearDown(self):
- """Remove test contents from disk."""
- LibrarianLayer.librarian_fixture.clear()
-
- def uploadPackage(self,
- changesfile="suite/bar_1.0-1/bar_1.0-1_source.changes"):
- """Helper function to upload a package."""
- with dbuser("uploader"):
- sync_policy = getPolicy(
- name='sync', distro='ubuntu', distroseries='breezy-autotest')
- bar_src = NascentUpload.from_changesfile_path(
- datadir(changesfile),
- sync_policy, DevNullLogger())
- bar_src.process()
- bar_src.do_accept()
- return bar_src
-
- def testBrokenAction(self):
- """Check if an unknown action raises CommandRunnerError."""
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'foo')
-
- def testHelpAction(self):
- """Check if help is working properly.
-
- Without arguments 'help' should return the docstring summary of
- all available actions.
-
- Optionally we can pass arguments corresponding to the specific
- actions we want to see the help, not available actions will be
- reported.
- """
- self.execute_command('help')
- self.assertEqual(
- ['Running: "help"',
- '\tinfo : Present the Queue item including its contents. ',
- '\taccept : Accept the contents of a queue item. ',
- '\treport : Present a report about the size of available '
- 'queues ',
- '\treject : Reject the contents of a queue item. ',
- '\toverride : Override information in a queue item content. ',
- '\tfetch : Fetch the contents of a queue item. '],
- self.test_output)
-
- self.execute_command('help fetch')
- self.assertEqual(
- ['Running: "help fetch"',
- '\tfetch : Fetch the contents of a queue item. '],
- self.test_output)
-
- self.execute_command('help foo')
- self.assertEqual(
- ['Running: "help foo"',
- 'Not available action(s): foo'],
- self.test_output)
-
- def testInfoAction(self):
- """Check INFO queue action without arguments present all items."""
- queue_action = self.execute_command('info')
- # check if the considered queue size matches the existent number
- # of records in sampledata
- bat = getUtility(IDistributionSet)['ubuntu']['breezy-autotest']
- queue_size = getUtility(IPackageUploadSet).count(
- status=PackageUploadStatus.NEW, distroseries=bat,
- pocket=PackagePublishingPocket.RELEASE)
- self.assertEqual(queue_size, queue_action.size)
- # check if none of them was filtered, since not filter term
- # was passed.
- self.assertEqual(queue_size, queue_action.items_size)
-
- def testInfoActionDoesNotSupportWildCards(self):
- """Check if an wildcard-like filter raises CommandRunnerError."""
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'info *')
-
- def testInfoActionByID(self):
- """Check INFO queue action filtering by ID.
-
- It should work as expected in case of existent ID in specified the
- location.
- Otherwise it raises CommandRunnerError if:
- * ID not found
- * specified ID doesn't match given suite name
- * specified ID doesn't match the queue name
- """
- queue_action = self.execute_command('info 1')
- # Check if only one item was retrieved.
- self.assertEqual(1, queue_action.items_size)
-
- displaynames = [item.displayname for item in queue_action.items]
- self.assertEqual(['mozilla-firefox'], displaynames)
-
- # Check passing multiple IDs.
- queue_action = self.execute_command('info 1 3 4')
- self.assertEqual(3, queue_action.items_size)
- [mozilla, netapplet, alsa] = queue_action.items
- self.assertEqual('mozilla-firefox', mozilla.displayname)
- self.assertEqual('netapplet', netapplet.displayname)
- self.assertEqual('alsa-utils', alsa.displayname)
-
- # Check not found ID.
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'info 100')
-
- # Check looking in the wrong suite.
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'info 1',
- suite_name='breezy-autotest-backports')
-
- # Check looking in the wrong queue.
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'info 1',
- queue_name='done')
-
- def testInfoActionByName(self):
- """Check INFO queue action filtering by name"""
- queue_action = self.execute_command('info pmount')
- # check if only one item was retrieved as expected in the current
- # sampledata
- self.assertEqual(1, queue_action.items_size)
-
- displaynames = [item.displayname for item in queue_action.items]
- self.assertEqual(['pmount'], displaynames)
-
- # Check looking for multiple names.
- queue_action = self.execute_command('info pmount alsa-utils')
- self.assertEqual(2, queue_action.items_size)
- [pmount, alsa] = queue_action.items
- self.assertEqual('pmount', pmount.displayname)
- self.assertEqual('alsa-utils', alsa.displayname)
-
- def testAcceptingSourceGeneratesEmail(self):
- """Check if accepting a source package generates an email."""
- # We need to upload a new source package to do this because the
- # sample data is horribly broken with published sources also in
- # the NEW queue. Doing it this way guarantees a nice set of data.
- self.uploadPackage()
-
- # Swallow email generated at the upload stage.
- stub.test_emails.pop()
-
- # Add a chroot to breezy-autotest/i386, so the system can create
- # builds for it.
- with lp_dbuser():
- a_file = getUtility(ILibraryFileAliasSet)[1]
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- breezy_autotest['i386'].addOrUpdateChroot(a_file)
-
- queue_action = self.execute_command(
- 'accept bar', no_mail=False)
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(2, len(stub.test_emails))
- # Emails sent are the announcement and the uploader's notification:
- self.assertEmail(['autotest_changes@xxxxxxxxxx'])
- self.assertEmail(
- ['Daniel Silverstone <daniel.silverstone@xxxxxxxxxxxxx>'])
-
- def testAcceptingSourceCreateBuilds(self):
- """Check if accepting a source package creates build records."""
- self.uploadPackage()
-
- # Swallow email generated at the upload stage.
- stub.test_emails.pop()
-
- # Add a chroot to breezy-autotest/i386, so the system can create
- # builds for it.
- with lp_dbuser():
- a_file = getUtility(ILibraryFileAliasSet)[1]
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- breezy_autotest['i386'].addOrUpdateChroot(a_file)
-
- queue_action = self.execute_command(
- 'accept bar', no_mail=False)
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(2, len(stub.test_emails))
-
- [queue_item] = queue_action.items
- [queue_source] = queue_item.sources
- sourcepackagerelease = queue_source.sourcepackagerelease
- [build] = sourcepackagerelease.builds
- self.assertEqual(
- 'i386 build of bar 1.0-1 in ubuntu breezy-autotest RELEASE',
- build.title)
- self.assertEqual(build.buildqueue_record.lastscore, 1755)
-
- def testAcceptingBinaryDoesntGenerateEmail(self):
- """Check if accepting a binary package does not generate email."""
- queue_action = self.execute_command(
- 'accept mozilla-firefox', no_mail=False)
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(0, len(stub.test_emails))
-
- def testAcceptingSourceClosesBug(self):
- """Check that accepting a source will close bugs appropriately."""
- # To speed up the publication process, single source uploads
- # are automatically published when they are accepted to avoid
- # another publisher cycle's worth of delay. When the source is
- # published, any bugs mentioned in the upload must be closed.
-
- # First we must upload the first version of 'bar' in Ubuntu Hoary.
- bar_src = self.uploadPackage()
- bar_src.queue_root.setAccepted()
- bar_src.queue_root.realiseUpload()
-
- # Now make a new bugtask for the "bar" package.
- with lp_dbuser():
- the_bug_id = 6
- bugtask_owner = getUtility(IPersonSet).getByName('kinnison')
- ubuntu = getUtility(IDistributionSet)['ubuntu']
- ubuntu_bar = ubuntu.getSourcePackage('bar')
- the_bug = getUtility(IBugSet).get(the_bug_id)
- bugtask = getUtility(IBugTaskSet).createTask(
- the_bug, bugtask_owner, ubuntu_bar)
-
- # The bugtask starts life as NEW.
- the_bug = getUtility(IBugSet).get(the_bug_id)
- bugtask = the_bug.getBugTask(ubuntu_bar)
- bug_status = bugtask.status.name
- self.assertEqual(
- bug_status, 'NEW',
- 'Bug status is %s, expected NEW' % bug_status)
-
- # Now, make an upload for the next version of "bar".
- bar2_src = self.uploadPackage(
- changesfile="suite/bar_1.0-2/bar_1.0-2_source.changes")
-
- # Now accept the new bar upload with the queue tool.
- self.execute_command('accept bar', no_mail=False)
-
- # The upload wants to close bug 6:
- bugs_fixed_header = bar2_src.changes._dict['Launchpad-bugs-fixed']
- self.assertEqual(
- bugs_fixed_header, str(the_bug_id),
- 'Expected bug %s in Launchpad-bugs-fixed, got %s'
- % (the_bug_id, bugs_fixed_header))
-
- # The upload should be in the DONE state:
- item_status = bar2_src.queue_root.status.name
- self.assertEqual(
- item_status, 'DONE',
- 'Upload status is %s, expected DONE' % item_status)
-
- # The bug should now be marked as fix released for the "bar"
- # bugtask:
- the_bug = getUtility(IBugSet).get(the_bug_id)
- bugtask = the_bug.getBugTask(ubuntu_bar)
- bug_status = bugtask.status.name
- self.assertEqual(
- bug_status, 'FIXRELEASED',
- 'Bug status is %s, expected FIXRELEASED')
-
- # Clean up.
- upload_data = datadir('suite/bar_1.0-2')
- os.remove(os.path.join(upload_data, 'bar_1.0.orig.tar.gz'))
-
- def testAcceptActionWithMultipleIDs(self):
- """Check if accepting multiple items at once works.
-
- We can specify multiple items to accept, even mixing IDs and names.
- e.g. queue accept alsa-utils 1 3
- """
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- queue_action = self.execute_command('accept 1 pmount 3')
-
- self.assertEqual(3, queue_action.items_size)
-
- self.assertQueueLength(1, breezy_autotest,
- PackageUploadStatus.ACCEPTED, u'mozilla-firefox')
- self.assertQueueLength(1, breezy_autotest,
- PackageUploadStatus.ACCEPTED, u'pmount')
- # Single-source upload went straight to DONE queue.
- self.assertQueueLength(1, breezy_autotest,
- PackageUploadStatus.DONE, u'netapplet')
-
- def testRemovedPublishRecordDoesNotAffectQueueNewness(self):
- """Check if REMOVED published record does not affect file NEWness.
-
- We only mark a file as *known* if there is a PUBLISHED record with
- the same name, other states like SUPERSEDED or REMOVED doesn't count.
-
- This is the case of 'pmount_0.1-1' in ubuntu/breezy-autotest/i386,
- there is a REMOVED publishing record for it as you can see in the
- first part of the test.
-
- Following we can see the correct presentation of the new flag ('N').
- Bug #59291
- """
- # inspect publishing history in sampledata for the suspicious binary
- # ensure is has a single entry and it is merked as REMOVED.
- ubuntu = getUtility(IDistributionSet)['ubuntu']
- bat_i386 = ubuntu['breezy-autotest']['i386']
- moz_publishing = bat_i386.getBinaryPackage('pmount').releases
-
- self.assertEqual(1, len(moz_publishing))
- self.assertEqual(PackagePublishingStatus.DELETED,
- moz_publishing[0].status)
-
- # invoke queue tool filtering by name
- queue_action = self.execute_command('info pmount')
-
- # ensure we retrived a single item
- self.assertEqual(1, queue_action.items_size)
-
- # and it is what we expect
- self.assertEqual('pmount', queue_action.items[0].displayname)
- self.assertEqual(moz_publishing[0].binarypackagerelease.build,
- queue_action.items[0].builds[0].build)
- # inspect output, note the presence of 'N' flag
- self.assertTrue(
- '| N pmount/0.1-1/i386' in '\n'.join(self.test_output))
-
- def testQueueSupportForSuiteNames(self):
- """Queue tool supports suite names properly.
-
- Two UNAPPROVED items are present for pocket RELEASE and only
- one for pocket UPDATES in breezy-autotest.
- Bug #59280
- """
- queue_action = self.execute_command(
- 'info', queue_name='unapproved',
- suite_name='breezy-autotest')
-
- self.assertEqual(2, queue_action.items_size)
- self.assertEqual(PackagePublishingPocket.RELEASE, queue_action.pocket)
-
- queue_action = self.execute_command(
- 'info', queue_name='unapproved',
- suite_name='breezy-autotest-updates')
-
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(PackagePublishingPocket.UPDATES, queue_action.pocket)
-
- def testQueueDoesNotAnnounceBackports(self):
- """Check if BACKPORTS acceptance are not announced publicly.
-
- Queue tool normally announce acceptance in the specified changeslist
- for the distroseries in question, however BACKPORTS announce doesn't
- fit very well in that list, they cause unwanted noise.
-
- Further details in bug #59443
- """
- with lp_dbuser():
- # Make breezy-autotest CURRENT in order to accept upload
- # to BACKPORTS.
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- breezy_autotest.status = SeriesStatus.CURRENT
-
- # Store the targeted queue item for future inspection.
- # Ensure it is what we expect.
- target_queue = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.UNAPPROVED,
- pocket=PackagePublishingPocket.BACKPORTS)[0]
- self.assertEqual(10, target_queue.id)
-
- # Ensure breezy-autotest is set.
- self.assertEqual(
- u'autotest_changes@xxxxxxxxxx', breezy_autotest.changeslist)
-
- # Accept the sampledata item.
- queue_action = self.execute_command(
- 'accept', queue_name='unapproved',
- suite_name='breezy-autotest-backports', no_mail=False)
-
- # Only one item considered.
- self.assertEqual(1, queue_action.items_size)
-
- # Previously stored reference should have new state now
- self.assertEqual('ACCEPTED', target_queue.status.name)
-
- # Only one email is sent to the changed-by email on the changes
- # file. No announcement email is sent.
- self.assertEqual(len(stub.test_emails), 1)
- self.assertEmail(
- ['Daniel Silverstone <daniel.silverstone@xxxxxxxxxxxxx>'])
-
- def testQueueDoesNotSendAnyEmailsForTranslations(self):
- """Check if no emails are sent when accepting translations.
-
- Queue tool should not send any emails to source uploads targeted to
- 'translation' section.
- They are the 'language-pack-*' and 'language-support-*' sources.
-
- Further details in bug #57708
- """
- with lp_dbuser():
- # Make breezy-autotest CURRENT in order to accept upload
- # to PROPOSED.
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- breezy_autotest.status = SeriesStatus.CURRENT
-
- # Store the targeted queue item for future inspection.
- # Ensure it is what we expect.
- target_queue = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.UNAPPROVED,
- pocket=PackagePublishingPocket.PROPOSED)[0]
- self.assertEqual(12, target_queue.id)
- source = target_queue.sources[0].sourcepackagerelease
- self.assertEqual('translations', source.section.name)
-
- # Accept the sampledata item.
- queue_action = self.execute_command(
- 'accept', queue_name='unapproved',
- suite_name='breezy-autotest-proposed', no_mail=False)
-
- # Only one item considered.
- self.assertEqual(1, queue_action.items_size)
-
- # Previously stored reference should have new state now.
- self.assertEqual('DONE', target_queue.status.name)
-
- # No email was sent.
- self.assertEqual(0, len(stub.test_emails))
-
- def assertQueueLength(self, expected_length, distro_series, status, name):
- queue_items = distro_series.getPackageUploads(
- status=status, name=name)
- self.assertEqual(expected_length, queue_items.count())
-
- def assertErrorAcceptingDuplicate(self):
- self.assertTrue(
- '** cnews could not be accepted due to '
- 'The source cnews - 1.0 is already accepted in ubuntu/'
- 'breezy-autotest and you cannot upload the same version '
- 'within the same distribution. You have to modify the source '
- 'version and re-upload.' in self.test_output)
-
- def testAcceptanceWorkflowForDuplications(self):
- """Check how queue tool behaves dealing with duplicated entries.
-
- Sampledata provides a duplication of cnews_1.0 in breezy-autotest
- UNAPPROVED queue.
-
- Step 1: executing 'accept cnews in unapproved queue' with duplicate
- cnews items in the UNAPPROVED queue, results in the oldest being
- accepted and the newer one remaining UNAPPROVED (and displaying
- an error about it to the user).
-
- Step 2: executing 'accept cnews in unapproved queue' with duplicate
- cnews items in the UNAPPROVED and ACCEPTED queues has no effect on
- the queues, and again displays an error to the user.
-
- Step 3: executing 'accept cnews in unapproved queue' with duplicate
- cnews items in the UNAPPROVED and DONE queues behaves the same as 2.
-
- Step 4: the remaining duplicated cnews item in UNAPPROVED queue can
- only be rejected.
- """
- with lp_dbuser():
- # Add a chroot to breezy-autotest/i386, so the system can create
- # builds for it.
- a_file = getUtility(ILibraryFileAliasSet)[1]
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- breezy_autotest['i386'].addOrUpdateChroot(a_file)
-
- # Certify we have a 'cnews' upload duplication in UNAPPROVED.
- self.assertQueueLength(
- 2, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
-
- # Step 1: try to accept both.
- self.execute_command(
- 'accept cnews', queue_name='unapproved',
- suite_name='breezy-autotest')
-
- # The first item, being a single source upload, is automatically
- # published when it's accepted.
- self.assertQueueLength(
- 1, breezy_autotest, PackageUploadStatus.DONE, u"cnews")
-
- # The last can't be accepted and remains in UNAPPROVED.
- self.assertErrorAcceptingDuplicate()
- self.assertQueueLength(
- 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
-
- # Step 2: try to accept the remaining item in UNAPPROVED.
- self.execute_command(
- 'accept cnews', queue_name='unapproved',
- suite_name='breezy-autotest')
- self.assertErrorAcceptingDuplicate()
- self.assertQueueLength(
- 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
-
- # Step 3: try to accept the remaining item in UNAPPROVED with the
- # duplication already in DONE.
- self.execute_command(
- 'accept cnews', queue_name='unapproved',
- suite_name='breezy-autotest')
- # It failed and te item remains in UNAPPROVED.
- self.assertErrorAcceptingDuplicate()
- self.assertQueueLength(
- 1, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
-
- # Step 4: The only possible destiny for the remaining item it REJECT.
- self.execute_command(
- 'reject cnews', queue_name='unapproved',
- suite_name='breezy-autotest')
- self.assertQueueLength(
- 0, breezy_autotest, PackageUploadStatus.UNAPPROVED, u"cnews")
- self.assertQueueLength(
- 1, breezy_autotest, PackageUploadStatus.REJECTED, u"cnews")
-
- def testRejectSourceSendsEmail(self):
- """Check that rejecting a source upload sends email."""
- queue_action = self.execute_command(
- 'reject alsa-utils', no_mail=False)
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(1, len(stub.test_emails))
- self.assertEmail(
- ['Daniel Silverstone <daniel.silverstone@xxxxxxxxxxxxx>'])
-
- def testRejectBinarySendsEmail(self):
- """Check that rejecting a binary upload sends email."""
- queue_action = self.execute_command('reject 2', no_mail=False)
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(1, len(stub.test_emails))
- self.assertEmail(
- ['Daniel Silverstone <daniel.silverstone@xxxxxxxxxxxxx>'])
-
- def testRejectLangpackSendsNoEmail(self):
- """Check that rejecting a language pack sends no email."""
- queue_action = self.execute_command(
- 'reject language-pack-de', queue_name='unapproved',
- suite_name='breezy-autotest-proposed')
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(0, len(stub.test_emails))
-
- def testRejectWithMultipleIDs(self):
- """Check if rejecting multiple items at once works.
-
- We can specify multiple items to reject, even mixing IDs and names.
- e.g. queue reject alsa-utils 1 3
- """
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
-
- # Run the command.
- queue_action = self.execute_command('reject 1 pmount 3')
-
- # Test what it did. Since all the queue items came out of the
- # NEW queue originally, the items processed should now be REJECTED.
- self.assertEqual(3, queue_action.items_size)
- self.assertQueueLength(1, breezy_autotest,
- PackageUploadStatus.REJECTED, u'mozilla-firefox')
- self.assertQueueLength(1, breezy_autotest,
- PackageUploadStatus.REJECTED, u'pmount')
- self.assertQueueLength(1, breezy_autotest,
- PackageUploadStatus.REJECTED, u'netapplet')
-
- def testOverrideSource(self):
- """Check if overriding sources works.
-
- We can specify multiple items to override, even mixing IDs and names.
- e.g. queue override source -c restricted alsa-utils 1 3
- """
- # Set up.
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
-
- # Basic operation overriding a single source 'alsa-utils' that
- # is currently main/base in the sample data.
- queue_action = self.execute_command('override source 4',
- component_name='restricted', section_name='web')
- self.assertEqual(1, queue_action.items_size)
- queue_item = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u"alsa-utils")[0]
- [source] = queue_item.sources
- self.assertEqual('restricted',
- source.sourcepackagerelease.component.name)
- self.assertEqual('web',
- source.sourcepackagerelease.section.name)
-
- # Override multiple sources at once and mix ID with name.
- queue_action = self.execute_command('override source 4 netapplet',
- component_name='universe', section_name='editors')
- # 'netapplet' appears 3 times, alsa-utils once.
- self.assertEqual(4, queue_action.items_size)
- self.assertEqual(2, queue_action.overrides_performed)
- # Check results.
- queue_items = list(breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'alsa-utils'))
- queue_items.extend(list(breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'netapplet')))
- for queue_item in queue_items:
- if queue_item.sources:
- [source] = queue_item.sources
- self.assertEqual('universe',
- source.sourcepackagerelease.component.name)
- self.assertEqual('editors',
- source.sourcepackagerelease.section.name)
-
- def testOverrideSourceWithArchiveChange(self):
- """Check if the archive changes as necessary on a source override.
-
- When overriding the component, the archive may change, so we check
- that here.
- """
- # Set up.
- ubuntu = getUtility(IDistributionSet)['ubuntu']
- breezy_autotest = ubuntu['breezy-autotest']
-
- # Test that it changes to partner when required.
- queue_action = self.execute_command('override source alsa-utils',
- component_name='partner')
- self.assertEqual(1, queue_action.items_size)
- [queue_item] = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u"alsa-utils")
- [source] = queue_item.sources
- self.assertEqual(source.sourcepackagerelease.upload_archive.purpose,
- ArchivePurpose.PARTNER)
-
- # Test that it changes back to primary when required.
- queue_action = self.execute_command('override source alsa-utils',
- component_name='main')
- self.assertEqual(1, queue_action.items_size)
- [queue_item] = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u"alsa-utils")
- [source] = queue_item.sources
- self.assertEqual(source.sourcepackagerelease.upload_archive.purpose,
- ArchivePurpose.PRIMARY)
-
- def testOverrideSourceWithNonexistentArchiveChange(self):
- """Check that overriding to a non-existent archive fails properly.
-
- When overriding the component, the archive may change to a
- non-existent one so ensure if fails.
- """
- with lp_dbuser():
- ubuntu = getUtility(IDistributionSet)['ubuntu']
- proxied_archive = getUtility(IArchiveSet).getByDistroPurpose(
- ubuntu, ArchivePurpose.PARTNER)
- comm_archive = removeSecurityProxy(proxied_archive)
- comm_archive.purpose = ArchivePurpose.PPA
-
- self.assertRaises(CommandRunnerError,
- self.execute_command,
- 'override source alsa-utils',
- component_name='partner')
-
- def testOverrideBinary(self):
- """Check if overriding binaries works.
-
- We can specify multiple items to override, even mixing IDs and names.
- e.g. queue override binary -c restricted alsa-utils 1 3
- """
- # Set up.
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
-
- # Override a binary, 'pmount', from its sample data of
- # main/base/IMPORTANT to restricted/web/extra.
- queue_action = self.execute_command('override binary pmount',
- component_name='restricted', section_name='web',
- priority_name='extra')
- self.assertEqual(1, queue_action.items_size)
- [queue_item] = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u"pmount")
- [packagebuild] = queue_item.builds
- for package in packagebuild.build.binarypackages:
- self.assertEqual('restricted', package.component.name)
- self.assertEqual('web', package.section.name)
- self.assertEqual('EXTRA', package.priority.name)
-
- # Override multiple binaries at once.
- queue_action = self.execute_command(
- 'override binary pmount mozilla-firefox',
- component_name='universe', section_name='editors',
- priority_name='optional')
- # Check results.
- self.assertEqual(2, queue_action.items_size)
- self.assertEqual(2, queue_action.overrides_performed)
- queue_items = list(breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'pmount'))
- queue_items.extend(list(breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'mozilla-firefox')))
- for queue_item in queue_items:
- [packagebuild] = queue_item.builds
- for package in packagebuild.build.binarypackages:
- self.assertEqual('universe', package.component.name)
- self.assertEqual('editors', package.section.name)
- self.assertEqual('OPTIONAL', package.priority.name)
-
- # Check that overriding by ID is warned to the user.
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'override binary 1',
- component_name='multiverse')
-
- def testOverridingMulipleBinariesFromSameBuild(self):
- """Check that multiple binary override works for the same build.
-
- Overriding binary packages generated from the same build should
- override each package individually.
- """
- # Start off by setting up a packageuploadbuild that points to
- # a build with two binaries.
- switch_dbuser("launchpad")
-
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
- [mozilla_queue_item] = breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'mozilla-firefox')
-
- # The build with ID '2' is for mozilla-firefox, which produces
- # binaries for 'mozilla-firefox' and 'mozilla-firefox-data'.
- PackageUploadBuild(packageupload=mozilla_queue_item, build=2)
-
- # Switching db users starts a new transaction. We must re-fetch
- # breezy-autotest.
- switch_dbuser("queued")
- breezy_autotest = getUtility(
- IDistributionSet)['ubuntu']['breezy-autotest']
-
- queue_action = self.execute_command(
- 'override binary mozilla-firefox-data mozilla-firefox',
- component_name='restricted', section_name='editors',
- priority_name='optional')
-
- # There are three binaries to override on this PackageUpload:
- # - mozilla-firefox in breezy-autotest
- # - mozilla-firefox and mozilla-firefox-data in warty
- # Each should be overridden exactly once.
- self.assertEqual(1, queue_action.items_size)
- self.assertEqual(3, queue_action.overrides_performed)
-
- queue_items = list(breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'mozilla-firefox-data'))
- queue_items.extend(list(breezy_autotest.getPackageUploads(
- status=PackageUploadStatus.NEW, name=u'mozilla-firefox')))
- for queue_item in queue_items:
- for packagebuild in queue_item.builds:
- for package in packagebuild.build.binarypackages:
- self.assertEqual(
- 'restricted', package.component.name,
- "The component '%s' is not the expected 'restricted'"
- "for package %s" % (
- package.component.name, package.name))
- self.assertEqual(
- 'editors', package.section.name,
- "The section '%s' is not the expected 'editors'"
- "for package %s" % (
- package.section.name, package.name))
- self.assertEqual(
- 'OPTIONAL', package.priority.name,
- "The priority '%s' is not the expected 'OPTIONAL'"
- "for package %s" % (
- package.section.name, package.name))
-
- def testOverrideBinaryWithArchiveChange(self):
- """Check if archive changes are disallowed for binary overrides.
-
- When overriding the component, the archive may change, so we check
- that here and make sure it's disallowed.
- """
- # Test that it changes to partner when required.
- self.assertRaises(
- CommandRunnerError, self.execute_command,
- 'override binary pmount', component_name='partner')
-
-
-class TestQueueActionLite(TestCaseWithFactory):
- """A lightweight unit test case for `QueueAction`.
-
- Meant for detailed tests that would be too expensive for full end-to-end
- tests.
- """
-
- layer = LaunchpadZopelessLayer
-
- def makeQueueAction(self, package_upload, distroseries=None,
- component=None, section=None,
- action_type=QueueAction):
- """Create a `QueueAction` for use with a `PackageUpload`.
-
- The action's `display` method is set to a `FakeMethod`.
- """
- if distroseries is None:
- distroseries = self.factory.makeDistroSeries(
- status=SeriesStatus.CURRENT,
- name="distroseriestestingpcjs")
- distro = distroseries.distribution
- if package_upload is None:
- package_upload = self.factory.makePackageUpload(
- distroseries=distroseries, archive=distro.main_archive)
- if component is None:
- component = self.factory.makeComponent()
- if section is None:
- section = self.factory.makeSection()
- queue = PackageUploadStatus.NEW
- priority_name = "STANDARD"
- display = FakeMethod()
- terms = ['*']
- return action_type(
- distro.name, distroseries.name, queue, terms, component.name,
- section.name, priority_name, display)
-
- def makeQueueActionOverride(self, package_upload, component, section,
- distroseries=None):
- return self.makeQueueAction(
- package_upload, distroseries, component, section,
- action_type=QueueActionOverride)
-
- def parseUploadSummaryLine(self, output_line):
- """Parse an output line from `QueueAction.displayItem`.
-
- :param output_line: A line of output text from `displayItem`.
- :return: A tuple of displayed items: (id, tag, name, version, age).
- """
- return tuple(item.strip() for item in output_line.split('|'))
-
- def test_display_actions_have_privileges_for_PackageCopyJob(self):
- # The methods that display uploads have privileges to work with
- # a PackageUpload that has a copy job.
- # Bundling tests for multiple operations into one test because
- # the database user change requires a costly commit.
- upload = self.factory.makeCopyJobPackageUpload()
- action = self.makeQueueAction(upload)
- switch_dbuser(config.uploadqueue.dbuser)
-
- action.displayItem(upload)
- self.assertNotEqual(0, action.display.call_count)
- action.display.calls = []
- action.displayInfo(upload)
- self.assertNotEqual(0, action.display.call_count)
-
- def test_accept_actions_have_privileges_for_PackageCopyJob(self):
- # The script also has privileges to approve uploads that have
- # copy jobs.
- distroseries = self.factory.makeDistroSeries(
- status=SeriesStatus.CURRENT)
- upload = self.factory.makeCopyJobPackageUpload(distroseries)
- switch_dbuser(config.uploadqueue.dbuser)
- upload.acceptFromQueue(DevNullLogger(), dry_run=True)
- # Flush changes to make sure we're not caching any updates that
- # the database won't allow. If this passes, we've got the
- # privileges.
- IStore(upload).flush()
-
- def test_displayItem_displays_PackageUpload_with_source(self):
- # displayItem can display a source package upload.
- upload = self.factory.makeSourcePackageUpload()
- action = self.makeQueueAction(upload)
-
- action.displayItem(upload)
-
- ((output, ), kwargs) = action.display.calls[0]
- (upload_id, tag, name, version, age) = self.parseUploadSummaryLine(
- output)
- self.assertEqual(str(upload.id), upload_id)
- self.assertEqual("S-", tag)
- self.assertThat(upload.displayname, StartsWith(name))
- self.assertThat(upload.package_version, StartsWith(version))
-
- def test_displayItem_displays_PackageUpload_with_PackageCopyJob(self):
- # displayItem can display a copy-job package upload.
- upload = self.factory.makeCopyJobPackageUpload()
- action = self.makeQueueAction(upload)
-
- action.displayItem(upload)
-
- ((output, ), kwargs) = action.display.calls[0]
- (upload_id, tag, name, version, age) = self.parseUploadSummaryLine(
- output)
- self.assertEqual(str(upload.id), upload_id)
- self.assertEqual("X-", tag)
- self.assertThat(upload.displayname, StartsWith(name))
- self.assertThat(upload.package_version, StartsWith(version))
-
- def test_override_works_with_PackageCopyJob(self):
- # "Sync" PackageUploads can be overridden just like sources,
- # test that here.
- new_component = self.factory.makeComponent()
- new_section = self.factory.makeSection()
- pocket = PackagePublishingPocket.RELEASE
- upload = self.factory.makeCopyJobPackageUpload(target_pocket=pocket)
- action = self.makeQueueActionOverride(
- upload, new_component, new_section,
- distroseries=upload.distroseries)
- # Patch this out because it uses data we don't have in the test;
- # it's unnecessary anyway.
- self.patch(action, "displayTitle", FakeMethod)
- action.terms = ["source", str(upload.id)]
- switch_dbuser(config.uploadqueue.dbuser)
- action.initialize()
- action.run()
-
- # Overriding a sync means putting the overrides in the job itself.
- self.assertEqual(
- new_component.name, upload.package_copy_job.component_name)
- self.assertEqual(
- new_section.name, upload.package_copy_job.section_name)
-
- def test_makeTag_returns_S_for_source_upload(self):
- upload = self.factory.makeSourcePackageUpload()
- self.assertEqual('S-', self.makeQueueAction(upload)._makeTag(upload))
-
- def test_makeTag_returns_B_for_binary_upload(self):
- upload = self.factory.makeBuildPackageUpload()
- self.assertEqual('-B', self.makeQueueAction(upload)._makeTag(upload))
-
- def test_makeTag_returns_SB_for_mixed_upload(self):
- upload = self.factory.makeSourcePackageUpload()
- upload.addBuild(self.factory.makeBinaryPackageBuild())
- self.assertEqual('SB', self.makeQueueAction(upload)._makeTag(upload))
-
- def test_makeTag_returns_X_for_copy_job_upload(self):
- upload = self.factory.makeCopyJobPackageUpload()
- self.assertEqual('X-', self.makeQueueAction(upload)._makeTag(upload))
-
- def test_makeTag_returns_dashes_for_custom_upload(self):
- upload = self.factory.makeCustomPackageUpload()
- self.assertEqual('--', self.makeQueueAction(upload)._makeTag(upload))
-
- def test_displayInfo_displays_PackageUpload_with_source(self):
- # displayInfo can display a source package upload.
- upload = self.factory.makeSourcePackageUpload()
- action = self.makeQueueAction(upload)
- action.displayInfo(upload)
- self.assertNotEqual(0, action.display.call_count)
-
- def test_displayInfo_displays_PackageUpload_with_PackageCopyJob(self):
- # displayInfo can display a copy-job package upload.
- upload = self.factory.makeCopyJobPackageUpload()
- action = self.makeQueueAction(upload)
- action.displayInfo(upload)
- self.assertNotEqual(0, action.display.call_count)
-
-
-class TestQueuePageClosingBugs(TestCaseWithFactory):
- # The distroseries +queue page can close bug when accepting
- # packages. Unit tests for that belong here.
-
- layer = DatabaseFunctionalLayer
-
- def test_close_bugs_for_sourcepackagerelease_with_private_bug(self):
- # lp.soyuz.scripts.processaccepted.close_bugs_for_sourcepackagerelease
- # should work with private bugs where the person using the queue
- # page doesn't have access to it.
- changes_file_template = "Format: 1.7\nLaunchpad-bugs-fixed: %s\n"
- # changelog_entry is required for an assertion inside the function
- # we're testing.
- spr = self.factory.makeSourcePackageRelease(changelog_entry="blah")
- archive_admin = self.factory.makePerson()
- bug = self.factory.makeBug(
- sourcepackagename=spr.sourcepackagename,
- distribution=spr.upload_distroseries.distribution,
- information_type=InformationType.USERDATA)
- changes = StringIO(changes_file_template % bug.id)
-
- with person_logged_in(archive_admin):
- # The archive admin user can't normally see this bug.
- self.assertRaises(ForbiddenAttribute, bug, 'status')
- # But the bug closure should work.
- close_bugs_for_sourcepackagerelease(spr, changes)
-
- # Verify it was closed.
- with celebrity_logged_in("admin"):
- self.assertEqual(
- bug.default_bugtask.status, BugTaskStatus.FIXRELEASED)
-
-
-class TestQueueToolInJail(TestQueueBase, TestCase):
- layer = LaunchpadZopelessLayer
- dbuser = config.uploadqueue.dbuser
-
- def setUp(self):
- """Create contents in disk for librarian sampledata.
-
- Setup and chdir into a temp directory, a jail, where we can
- control the file creation properly
- """
- fillLibrarianFile(1, content='One')
- fillLibrarianFile(52, content='Fifty-Two')
- self._home = os.path.abspath('')
- self._jail = tempfile.mkdtemp()
- os.chdir(self._jail)
- TestQueueBase.setUp(self)
-
- def tearDown(self):
- """Remove test contents from disk.
-
- chdir back to the previous path (home) and remove the temp
- directory used as jail.
- """
- os.chdir(self._home)
- LibrarianLayer.librarian_fixture.clear()
- shutil.rmtree(self._jail)
-
- def _listfiles(self):
- """Return a list of files present in jail."""
- return os.listdir(self._jail)
-
- def _getsha1(self, filename):
- """Return a sha1 hex digest of a file"""
- file_sha = hashlib.sha1()
- opened_file = open(filename, "r")
- for chunk in filechunks(opened_file):
- file_sha.update(chunk)
- opened_file.close()
- return file_sha.hexdigest()
-
- def testFetchActionByIDDoNotOverwriteFilesystem(self):
- """Check if queue fetch action doesn't overwrite files.
-
- Since we allow existence of duplications in NEW and UNAPPROVED
- queues, we are able to fetch files from queue items and they'd
- get overwritten causing obscure problems.
-
- Instead of overwrite a file in the working directory queue will
- fail, raising a CommandRunnerError.
-
- bug 67014: Don't complain if files are the same
- """
- self.execute_command('fetch 1')
- self.assertEqual(
- ['mozilla-firefox_0.9_i386.changes'], self._listfiles())
-
- # checksum the existing file
- existing_sha1 = self._getsha1(self._listfiles()[0])
-
- # fetch will NOT raise and not overwrite the file in disk
- self.execute_command('fetch 1')
-
- # checksum file again
- new_sha1 = self._getsha1(self._listfiles()[0])
-
- # Check that the file has not changed (we don't care if it was
- # re-written, just that it's not changed)
- self.assertEqual(existing_sha1, new_sha1)
-
- def testFetchActionRaisesErrorIfDifferentFileAlreadyFetched(self):
- """Check that fetching a file that has already been fetched
- raises an error if they are not the same file. (bug 67014)
- """
- CLOBBERED = "you're clobbered"
-
- self.execute_command('fetch 1')
- self.assertEqual(
- ['mozilla-firefox_0.9_i386.changes'], self._listfiles())
-
- # clobber the existing file, fetch it again and expect an exception
- f = open(self._listfiles()[0], "w")
- f.write(CLOBBERED)
- f.close()
-
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'fetch 1')
-
- # make sure the file has not changed
- f = open(self._listfiles()[0], "r")
- line = f.read()
- f.close()
-
- self.assertEqual(CLOBBERED, line)
-
- def testFetchActionByNameDoNotOverwriteFilesystem(self):
- """Same as testFetchActionByIDDoNotOverwriteFilesystem
-
- The sampledata provides duplicated 'cnews' entries, filesystem
- conflict will happen inside the same batch,
-
- Queue will fetch the oldest and raise.
- """
- self.assertRaises(
- CommandRunnerError, self.execute_command, 'fetch cnews',
- queue_name='unapproved', suite_name='breezy-autotest')
-
- self.assertEqual(['netapplet-1.0.0.tar.gz'], self._listfiles())
-
- def testQueueFetch(self):
- """Check that a basic fetch operation works."""
- FAKE_CHANGESFILE_CONTENT = "Fake Changesfile"
- FAKE_DEB_CONTENT = "Fake DEB"
- fillLibrarianFile(1, FAKE_CHANGESFILE_CONTENT)
- fillLibrarianFile(90, FAKE_DEB_CONTENT)
- self.execute_command('fetch pmount')
-
- # Check the files' names.
- files = sorted(self._listfiles())
- self.assertEqual(
- ['netapplet-1.0.0.tar.gz', 'pmount_1.0-1_all.deb'],
- files)
-
- # Check the files' contents.
- changes_file = open('netapplet-1.0.0.tar.gz')
- self.assertEqual(changes_file.read(), FAKE_CHANGESFILE_CONTENT)
- changes_file.close()
- debfile = open('pmount_1.0-1_all.deb')
- self.assertEqual(debfile.read(), FAKE_DEB_CONTENT)
- debfile.close()
-
- def testFetchMultipleItems(self):
- """Check if fetching multiple items at once works.
-
- We can specify multiple items to fetch, even mixing IDs and names.
- e.g. queue fetch alsa-utils 1 3
- """
- self.execute_command('fetch 3 mozilla-firefox')
- files = self._listfiles()
- files.sort()
- self.assertEqual(
- ['mozilla-firefox_0.9_i386.changes', 'netapplet-1.0.0.tar.gz'],
- files)
-
- def testFetchWithoutChanges(self):
- """Check that fetch works without a changes file (eg. from gina)."""
- pus = getUtility(IDistributionSet).getByName('ubuntu').getSeries(
- 'breezy-autotest').getPackageUploads(name=u'pmount')
- for pu in pus:
- removeSecurityProxy(pu).changesfile = None
-
- FAKE_DEB_CONTENT = "Fake DEB"
- fillLibrarianFile(90, FAKE_DEB_CONTENT)
- self.execute_command('fetch pmount')
-
- # Check the files' names.
- files = sorted(self._listfiles())
- self.assertEqual(['pmount_1.0-1_all.deb'], files)
=== modified file 'lib/lp/soyuz/tests/test_packageupload.py'
--- lib/lp/soyuz/tests/test_packageupload.py 2012-07-11 13:43:32 +0000
+++ lib/lp/soyuz/tests/test_packageupload.py 2012-07-11 16:34:21 +0000
@@ -12,6 +12,7 @@
urlopen,
)
+from debian.deb822 import Changes
from lazr.restfulclient.errors import (
BadRequest,
Unauthorized,
@@ -37,6 +38,7 @@
from lp.services.librarian.browser import ProxiedLibraryFileAlias
from lp.services.log.logger import BufferLogger
from lp.services.mail import stub
+from lp.services.mail.sendmail import format_address_for_person
from lp.soyuz.adapters.overrides import SourceOverride
from lp.soyuz.enums import (
ArchivePurpose,
@@ -412,6 +414,203 @@
self.assertEqual(current_component, spr.component)
self.assertEqual(new_section, spr.section)
+ def makeSourcePackageUpload(self, pocket=None, sourcepackagename=None,
+ section_name=None, changes_dict=None):
+ """Make a useful source package upload for queue tests."""
+ distroseries = self.test_publisher.distroseries
+ distroseries.changeslist = "autotest_changes@xxxxxxxxxx"
+ uploader = self.factory.makePerson()
+ key = self.factory.makeGPGKey(owner=uploader)
+ changes = Changes({"Changed-By": uploader.preferredemail.email})
+ if changes_dict is not None:
+ changes.update(changes_dict)
+ upload = self.factory.makePackageUpload(
+ archive=distroseries.main_archive, distroseries=distroseries,
+ pocket=pocket, changes_file_content=changes.dump().encode("UTF-8"),
+ signing_key=key)
+ spr = self.factory.makeSourcePackageRelease(
+ sourcepackagename=sourcepackagename, distroseries=distroseries,
+ component="main", section_name=section_name,
+ changelog_entry="dummy")
+ upload.addSource(spr)
+ spr.addFile(self.factory.makeLibraryFileAlias(
+ filename="%s_%s.dsc" % (spr.name, spr.version)))
+ transaction.commit()
+ return upload, uploader
+
+ def makeBuildPackageUpload(self):
+ """Make a useful build package upload for queue tests."""
+ distroseries = self.test_publisher.distroseries
+ distroseries.changeslist = "autotest_changes@xxxxxxxxxx"
+ uploader = self.factory.makePerson()
+ key = self.factory.makeGPGKey(owner=uploader)
+ changes = Changes({"Changed-By": uploader.preferredemail.email})
+ upload = self.factory.makePackageUpload(
+ archive=distroseries.main_archive, distroseries=distroseries,
+ changes_file_content=changes.dump().encode("UTF-8"),
+ signing_key=key)
+ build = self.factory.makeBinaryPackageBuild(
+ distroarchseries=self.test_publisher.breezy_autotest_i386)
+ upload.addBuild(build)
+ bpr = self.factory.makeBinaryPackageRelease(build=build)
+ bpr.addFile(self.factory.makeLibraryFileAlias(
+ filename="%s_%s_i386.deb" % (bpr.name, bpr.version)))
+ transaction.commit()
+ return upload, uploader
+
+ def assertEmail(self, expected_to_addrs):
+ """Pop an email from the stub queue and check its recipients."""
+ _, to_addrs, _ = stub.test_emails.pop()
+ self.assertEqual(expected_to_addrs, to_addrs)
+
+ def test_acceptFromQueue_source_sends_email(self):
+ # Accepting a source package sends emails to the announcement list
+ # and the uploader.
+ self.test_publisher.prepareBreezyAutotest()
+ upload, uploader = self.makeSourcePackageUpload()
+ upload.acceptFromQueue()
+ self.assertEqual(2, len(stub.test_emails))
+ # Emails sent are the announcement and the uploader's notification:
+ self.assertEmail(["autotest_changes@xxxxxxxxxx"])
+ self.assertEmail([format_address_for_person(uploader)])
+
+ def test_acceptFromQueue_source_backports_sends_no_announcement(self):
+ # Accepting a source package into BACKPORTS does not send an
+ # announcement email to the distroseries changeslist (see bug
+ # #59443). It still sends an acknowledgement to the uploader.
+ self.test_publisher.prepareBreezyAutotest()
+ self.test_publisher.distroseries.status = SeriesStatus.CURRENT
+ upload, uploader = self.makeSourcePackageUpload(
+ pocket=PackagePublishingPocket.BACKPORTS)
+ upload.acceptFromQueue()
+ self.assertEqual(1, len(stub.test_emails))
+ # Only one email is sent, to the person in the changed-by field. No
+ # announcement email is sent.
+ self.assertEmail([format_address_for_person(uploader)])
+
+ def test_acceptFromQueue_source_translations_sends_no_email(self):
+ # Accepting source packages in the "translations" section (i.e.
+ # language packs) does not send any email. See bug #57708.
+ self.test_publisher.prepareBreezyAutotest()
+ self.test_publisher.distroseries.status = SeriesStatus.CURRENT
+ upload, _ = self.makeSourcePackageUpload(
+ pocket=PackagePublishingPocket.PROPOSED,
+ section_name="translations")
+ upload.acceptFromQueue()
+ self.assertEqual("DONE", upload.status.name)
+ self.assertEqual(0, len(stub.test_emails))
+
+ def test_acceptFromQueue_source_creates_builds(self):
+ # Accepting a source package creates build records.
+ self.test_publisher.prepareBreezyAutotest()
+ upload, _ = self.makeSourcePackageUpload()
+ upload.acceptFromQueue()
+ spr = upload.sourcepackagerelease
+ [build] = spr.builds
+ self.assertEqual(
+ "i386 build of %s %s in ubuntutest breezy-autotest RELEASE" % (
+ spr.name, spr.version),
+ build.title)
+
+ def test_acceptFromQueue_source_closes_bug(self):
+ # Accepting a source package closes bugs appropriately.
+ self.test_publisher.prepareBreezyAutotest()
+
+ # Upload the first version of a package.
+ upload_one, _ = self.makeSourcePackageUpload()
+ upload_one.setAccepted()
+ upload_one.realiseUpload()
+ spr = upload_one.sourcepackagerelease
+
+ # Make a new bug task for this package. It starts life as NEW.
+ dsp = self.test_publisher.ubuntutest.getSourcePackage(spr.name)
+ task = self.factory.makeBugTask(target=dsp, publish=False)
+ self.assertEqual("NEW", task.status.name)
+
+ # Upload the next version of the same package, closing this bug.
+ changes = Changes({"Launchpad-Bugs-Fixed": str(task.bug.id)})
+ upload_two, _ = self.makeSourcePackageUpload(
+ sourcepackagename=spr.sourcepackagename, changes_dict=changes)
+
+ # Accept the new upload. It should reach the DONE state, and should
+ # close the bug.
+ upload_two.acceptFromQueue()
+ self.assertEqual("DONE", upload_two.status.name)
+ self.assertEqual("FIXRELEASED", task.status.name)
+
+ def test_acceptFromQueue_binary_sends_no_email(self):
+ # Accepting a binary package does not send email.
+ self.test_publisher.prepareBreezyAutotest()
+ upload, _ = self.makeBuildPackageUpload()
+ upload.acceptFromQueue()
+ self.assertEqual(0, len(stub.test_emails))
+
+ def test_acceptFromQueue_handles_duplicates(self):
+ # Duplicate queue entries are handled sensibly.
+ self.test_publisher.prepareBreezyAutotest()
+ distroseries = self.test_publisher.distroseries
+ upload_one = self.factory.makePackageUpload(
+ archive=distroseries.main_archive, distroseries=distroseries)
+ upload_one.addSource(self.factory.makeSourcePackageRelease(
+ sourcepackagename="cnews", distroseries=distroseries,
+ component="main", version="1.0"))
+ upload_two = self.factory.makePackageUpload(
+ archive=distroseries.main_archive, distroseries=distroseries)
+ upload_two.addSource(self.factory.makeSourcePackageRelease(
+ sourcepackagename="cnews", distroseries=distroseries,
+ component="main", version="1.0"))
+ transaction.commit()
+ upload_one.setUnapproved()
+ upload_one.syncUpdate()
+ upload_two.setUnapproved()
+ upload_two.syncUpdate()
+
+ # There are now duplicate uploads in UNAPPROVED.
+ unapproved = distroseries.getPackageUploads(
+ status=PackageUploadStatus.UNAPPROVED, name=u"cnews")
+ self.assertEqual(2, unapproved.count())
+
+ # Accepting one of them works. (Since it's a single source upload,
+ # it goes straight to DONE.)
+ upload_one.acceptFromQueue()
+ self.assertEqual("DONE", upload_one.status.name)
+ transaction.commit()
+
+ # Trying to accept the second fails.
+ self.assertRaises(
+ QueueInconsistentStateError, upload_two.acceptFromQueue)
+ self.assertEqual("UNAPPROVED", upload_two.status.name)
+
+ # Rejecting the second upload works.
+ upload_two.rejectFromQueue()
+ self.assertEqual("REJECTED", upload_two.status.name)
+
+ def test_rejectFromQueue_source_sends_email(self):
+ # Rejecting a source package sends an email to the uploader.
+ self.test_publisher.prepareBreezyAutotest()
+ upload, uploader = self.makeSourcePackageUpload()
+ upload.rejectFromQueue()
+ self.assertEqual(1, len(stub.test_emails))
+ self.assertEmail([format_address_for_person(uploader)])
+
+ def test_rejectFromQueue_binary_sends_email(self):
+ # Rejecting a binary package sends an email to the uploader.
+ self.test_publisher.prepareBreezyAutotest()
+ upload, uploader = self.makeBuildPackageUpload()
+ upload.rejectFromQueue()
+ self.assertEqual(1, len(stub.test_emails))
+ self.assertEmail([format_address_for_person(uploader)])
+
+ def test_rejectFromQueue_source_translations_sends_no_email(self):
+ # Rejecting a language pack sends no email.
+ self.test_publisher.prepareBreezyAutotest()
+ self.test_publisher.distroseries.status = SeriesStatus.CURRENT
+ upload, _ = self.makeSourcePackageUpload(
+ pocket=PackagePublishingPocket.PROPOSED,
+ section_name="translations")
+ upload.rejectFromQueue()
+ self.assertEqual(0, len(stub.test_emails))
+
class TestPackageUploadPrivacy(TestCaseWithFactory):
"""Test PackageUpload security."""
=== removed file 'scripts/ftpmaster-tools/queue'
--- scripts/ftpmaster-tools/queue 2012-06-29 08:40:05 +0000
+++ scripts/ftpmaster-tools/queue 1970-01-01 00:00:00 +0000
@@ -1,125 +0,0 @@
-#!/usr/bin/python -S
-#
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Queue management script
-
-Tool for handling and visualisation of upload queue records.
-"""
-
-import _pythonpath
-
-import transaction
-
-from lp.services.config import config
-from lp.services.scripts.base import (
- LaunchpadScript,
- LaunchpadScriptFailure,
- )
-from lp.soyuz.scripts.queue import (
- CommandRunner,
- CommandRunnerError,
- name_queue_map,
- )
-
-
-class QueueScript(LaunchpadScript):
-
- usage = 'Usage: %prog [options] <command>'
-
- def add_my_options(self):
- self.parser.add_option(
- "-Q", "--queue",
- dest="queue_name", metavar="QUEUE", default="new",
- help="Which queue to consider")
-
- self.parser.add_option(
- "-d", "--distribution",
- dest="distribution_name", metavar="DISTRO", default=None,
- help="Which distro to look in")
-
- self.parser.add_option(
- "-s", "--suite",
- dest="suite_name", metavar="DISTRORELEASE", default=None,
- help=("Which distrorelease to look in, defaults "
- "to distribution 'currentseries'."))
-
- self.parser.add_option(
- "-N", "--dry-run", action="store_true",
- dest="dryrun", metavar="DRY_RUN", default=False,
- help="Whether to treat this as a dry-run or not.")
-
- self.parser.add_option(
- "-M", "--no-mail", action="store_true",
- dest="nomail", metavar="NO_MAIL", default=False,
- help="Whether to send announce email or not.")
-
- self.parser.add_option(
- "-e", "--exact-match", action="store_true",
- dest="exact_match", metavar="EXACTMATCH", default=False,
- help="Whether treat filter as a exact match or not.")
-
- self.parser.add_option(
- "-i", "--ignore-errors", action="store_true",
- dest="ignore_errors", metavar="IGNOREERRORS", default=False,
- help="Ignore errors when performing a list of commands.")
-
- self.parser.add_option(
- "-f", "--file", metavar="FILE", default=None,
- help="file containing a sequence of command lines.")
-
- self.parser.add_option(
- "-c", "--component", dest="component_name",
- metavar="COMPONENT", default=None,
- help="When overriding, move package to COMPONENT")
-
- self.parser.add_option(
- "-x", "--section", dest="section_name",
- metavar="SECTION", default=None,
- help="When overriding, move package to SECTION")
-
- self.parser.add_option(
- "-p", "--priority", dest="priority_name",
- metavar="PRIORITY", default=None,
- help="When overriding, move package to PRIORITY")
-
- def main(self):
- if self.options.queue_name not in name_queue_map:
- self.parser.error(
- 'Unable to map queue name "%s"' % self.options.queue_name)
-
- no_mail = self.options.dryrun or self.options.nomail
- queue = name_queue_map[self.options.queue_name]
-
- if self.options.file:
- args_list = [self.args.strip().split() for args in
- open(self.options.file).readlines()]
- else:
- args_list = [self.args]
-
- cmd_runner = CommandRunner(
- queue, self.options.distribution_name, self.options.suite_name,
- no_mail, self.options.component_name, self.options.section_name,
- self.options.priority_name, log=self.logger)
-
- print "Initializing connection to queue %s" % self.options.queue_name
-
- for single_args in args_list:
- try:
- cmd_runner.execute(single_args, self.options.exact_match)
- except CommandRunnerError as info:
- print (info)
- if self.options.ignore_errors:
- continue
- transaction.abort()
- raise LaunchpadScriptFailure(
- 'Error encountered -- aborting current transaction')
- else:
- if not self.options.dryrun:
- transaction.commit()
- else:
- print "DRY RUN requested, not committing."
-
-if __name__ == '__main__':
- QueueScript('queue', config.uploadqueue.dbuser).run()
Follow ups
