launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #05425
[Merge] lp:~jtv/launchpad/bug-884649-branch-4 into lp:launchpad
Jeroen T. Vermeulen has proposed merging lp:~jtv/launchpad/bug-884649-branch-4 into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #884649 in Launchpad itself: "Ubuntu publisher is taking more than an hour to complete"
https://bugs.launchpad.net/launchpad/+bug/884649
For more details, see:
https://code.launchpad.net/~jtv/launchpad/bug-884649-branch-4/+merge/81265
= Summary =
The second pass of binary package domination has been taking far too long. It repeatedly queried for active architecture-specific binary package publications on given source package releases. This was a necessary change that we chose to implement naïvely-but-correctly at first, as per Knuth's Law.
Generally, the query will be repeated for the same source package release across all architectures, with (after some other branches I landed for this bug) the same answer every time.
== Proposed fix ==
This branch picks the fruit from the tree planted and watered in the preceding branches for the same bug. It caches the result of that repetitive query between iterations.
== Pre-implementation notes ==
See the notes on bug 884649 for the plan we're following here. A few more low-risk optimizations remain to be added after this branch, if necessary:
* More bulk-loading of references to BinaryPackageBuild, SourcePackageRelease etc.
* As Raphaël suggested: use BinaryPackagePublishingHistory.binarypackagename now that it's finally been fully initialized.
== Implementation details ==
It might have been possible to pre-populate the entire cache in one huge query. Surprisingly, memory footprint is probably not a limiting factor there. But now that the second domination pass skips packages where it's known that it won't do any good, it might be costly to figure out in advance exactly for which SourcePackageReleases the information would be needed.
I replaced BinaryPackagePublishingHistory.getOtherPublicationsForSameSource with SourcePackageRelease.getActiveArchSPecificPublications. Quite a mouthful, eh? The change followed naturally from extracting the part where we find the relevant SourcePackageRelease. The BPPH itself isn't relevant as such; only a few of its attributes are relevant as search criteria, and those are now passed as parameters.
== Tests ==
{{{
./bin/test -vvc lp.archivepublisher.tests.test_dominator
./bin/test -vvc lp.soyuz.tests.test_publishing
./bin/test -vvc lp.soyuz.tests.test_sourcepackagerelease
}}}
== Demo and Q/A ==
Upload & process a few revisions of a source package with both arch-specific and arch-independent binary packages. Maybe some pure arch-specific and arch-indep ones as well.
Sabotage a binary package for one architecture: reset the BPPH for one of the arch-specific binary packages (from a mixed source package) to Pending.
Run the dominator.
See that it functioned normally:
* Of the architecture-dependent binary packages, all revisions are superseded except for the latest revision of each.
* For the architecture you “sabotaged,” the last Published BPPH stays Published and the Pending one stays Pending.
* Architecture-independent BPPHs are Superseded, except for the very latest revision of each _and_ except for the revision matching the currently Published revision of the binary package you sabotaged.
In other words, there should be two live versions of that arch-indep binary package. Its other releases should be superseded as normal though.
= Launchpad lint =
One pre-existing lint item that seems to be just cleverness confusing the linter. I'm not touching it.
Checking for conflicts and issues in changed files.
Linting changed files:
lib/lp/archivepublisher/domination.py
lib/lp/soyuz/tests/test_publishing.py
lib/lp/soyuz/model/publishing.py
lib/lp/soyuz/tests/test_sourcepackagerelease.py
lib/lp/soyuz/model/sourcepackagerelease.py
lib/lp/archivepublisher/tests/test_dominator.py
./lib/lp/soyuz/model/sourcepackagerelease.py
204: redefinition of function 'copyright' from line 195
--
https://code.launchpad.net/~jtv/launchpad/bug-884649-branch-4/+merge/81265
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~jtv/launchpad/bug-884649-branch-4 into lp:launchpad.
=== modified file 'lib/lp/archivepublisher/domination.py'
--- lib/lp/archivepublisher/domination.py 2011-11-04 09:47:37 +0000
+++ lib/lp/archivepublisher/domination.py 2011-11-04 12:37:24 +0000
@@ -216,19 +216,19 @@
return publications
-def find_live_source_versions(publications):
- """Find versions out of Published `publications` that should stay live.
+def find_live_source_versions(sorted_pubs):
+ """Find versions out of Published publications that should stay live.
This particular notion of liveness applies to source domination: the
latest version stays live, and that's it.
- :param publications: An iterable of `SourcePackagePublishingHistory`
+ :param sorted_pubs: An iterable of `SourcePackagePublishingHistory`
sorted by descending package version.
:return: A list of live versions.
"""
# Given the required sort order, the latest version is at the head
# of the list.
- return [publications[0].sourcepackagerelease.version]
+ return [sorted_pubs[0].sourcepackagerelease.version]
def get_binary_versions(binary_publications):
@@ -241,26 +241,73 @@
return [pub.binarypackagerelease.version for pub in binary_publications]
-def find_live_binary_versions_pass_1(publications):
+def find_live_binary_versions_pass_1(sorted_pubs):
"""Find versions out of Published `publications` that should stay live.
This particular notion of liveness applies to first-pass binary
domination: the latest version stays live, and so do publications of
binary packages for the "all" architecture.
- :param publications: An iterable of `BinaryPackagePublishingHistory`,
+ :param sorted_pubs: An iterable of `BinaryPackagePublishingHistory`,
sorted by descending package version.
:return: A list of live versions.
"""
- publications = list(publications)
- latest = publications.pop(0)
+ sorted_pubs = list(sorted_pubs)
+ latest = sorted_pubs.pop(0)
return get_binary_versions(
[latest] + [
- pub for pub in publications if not pub.architecture_specific])
-
-
-def find_live_binary_versions_pass_2(publications):
- """Find versions out of Published `publications` that should stay live.
+ pub for pub in sorted_pubs if not pub.architecture_specific])
+
+
+class ArchSpecificPublicationsCache:
+ """Cache to track which releases have arch-specific publications.
+
+ This is used for second-pass binary domination:
+ architecture-independent binary publications cannot be superseded as long
+ as any architecture-dependent binary publications built from the same
+ source package release are still active. Thus such arch-indep
+ publications are reprieved from domination.
+
+ This class looks up whether there are any. That only needs to be looked
+ up in the database once per (source package release, archive,
+ distroseries, pocket). Hence this cache.
+ """
+ def __init__(self):
+ self.cache = {}
+
+ @staticmethod
+ def getKey(bpph):
+ """Extract just the relevant bits of information from a bpph."""
+ return (
+ bpph.binarypackagerelease.build.source_package_release,
+ bpph.archive,
+ bpph.distroseries,
+ bpph.pocket,
+ )
+
+ def hasArchSpecificPublications(self, bpph):
+ """Does bpph have active, arch-specific publications?
+
+ If so, the dominator will want to reprieve `bpph`.
+ """
+ assert not bpph.architecture_specific, (
+ "Wrongly dominating arch-specific binary pub in pass 2.")
+
+ key = self.getKey(bpph)
+ if key not in self.cache:
+ self.cache[key] = self._lookUp(*key)
+ return self.cache[key]
+
+ @staticmethod
+ def _lookUp(spr, archive, distroseries, pocket):
+ """Look up an answer in the database."""
+ query = spr.getActiveArchSpecificPublications(
+ archive, distroseries, pocket)
+ return not query.is_empty()
+
+
+def find_live_binary_versions_pass_2(sorted_pubs, cache):
+ """Find versions out of Published publications that should stay live.
This particular notion of liveness applies to second-pass binary
domination: the latest version stays live, and architecture-specific
@@ -282,23 +329,23 @@
domination code confuses matters by using the term "active" to mean only
Published publications).
- :param publications: An iterable of `BinaryPackagePublishingHistory`,
+ :param sorted_pubs: An iterable of `BinaryPackagePublishingHistory`,
sorted by descending package version.
+ :param cache: An `ArchSpecificPublicationsCache` to reduce the number of
+ times we need to look up whether an spr/archive/distroseries/pocket
+ has active arch-specific publications.
:return: A list of live versions.
"""
- publications = list(publications)
- latest = publications.pop(0)
+ sorted_pubs = list(sorted_pubs)
+ latest = sorted_pubs.pop(0)
is_arch_specific = attrgetter('architecture_specific')
- arch_specific_pubs = list(ifilter(is_arch_specific, publications))
- arch_indep_pubs = list(ifilterfalse(is_arch_specific, publications))
+ arch_specific_pubs = list(ifilter(is_arch_specific, sorted_pubs))
+ arch_indep_pubs = list(ifilterfalse(is_arch_specific, sorted_pubs))
- # XXX JeroenVermeulen 2011-11-01 bug=884649: This is likely to be
- # costly, and the result could be reused for all builds of the same
- # source package release to all architectures.
reprieved_pubs = [
pub
for pub in arch_indep_pubs
- if pub.getOtherPublicationsForSameSource().any()]
+ if cache.hasArchSpecificPublications(pub)]
return get_binary_versions([latest] + arch_specific_pubs + reprieved_pubs)
@@ -324,7 +371,7 @@
self.logger = logger
self.archive = archive
- def dominatePackage(self, publications, live_versions, generalization):
+ def dominatePackage(self, sorted_pubs, live_versions, generalization):
"""Dominate publications for a single package.
The latest publication for any version in `live_versions` stays
@@ -339,7 +386,7 @@
previous latest version of a package has disappeared from the Sources
list we import.
- :param publications: Iterable of publications for the same package,
+ :param sorted_pubs: A list of publications for the same package,
in the same archive, series, and pocket, all with status
`PackagePublishingStatus.PUBLISHED`. They must be sorted from
most current to least current, as would be the result of
@@ -357,7 +404,7 @@
self.logger.debug(
"Package has %d live publication(s). Live versions: %s",
- len(publications), live_versions)
+ len(sorted_pubs), live_versions)
# Verify that the publications are really sorted properly.
check_order = OrderingCheck(cmp=generalization.compare, reverse=True)
@@ -365,7 +412,7 @@
current_dominant = None
dominant_version = None
- for pub in publications:
+ for pub in sorted_pubs:
check_order.check(pub)
version = generalization.getPackageVersion(pub)
@@ -619,6 +666,7 @@
# (In maintaining this code, bear in mind that some or all of a
# source package's binary packages may switch between
# arch-specific and arch-indep between releases.)
+ reprieve_cache = ArchSpecificPublicationsCache()
for distroarchseries in distroseries.architectures:
self.logger.info("Finding binaries...(2nd pass)")
bins = self.findBinariesForDomination(distroarchseries, pocket)
@@ -628,7 +676,8 @@
pubs = sorted_packages[name]
self.logger.debug("Dominating %s" % name)
assert len(pubs) > 0, "Dominating zero binaries in 2nd pass!"
- live_versions = find_live_binary_versions_pass_2(pubs)
+ live_versions = find_live_binary_versions_pass_2(
+ pubs, reprieve_cache)
self.dominatePackage(pubs, live_versions, generalization)
def _composeActiveSourcePubsCondition(self, distroseries, pocket):
=== modified file 'lib/lp/archivepublisher/tests/test_dominator.py'
--- lib/lp/archivepublisher/tests/test_dominator.py 2011-11-04 09:47:37 +0000
+++ lib/lp/archivepublisher/tests/test_dominator.py 2011-11-04 12:37:24 +0000
@@ -15,6 +15,7 @@
from canonical.database.sqlbase import flush_database_updates
from canonical.testing.layers import ZopelessDatabaseLayer
from lp.archivepublisher.domination import (
+ ArchSpecificPublicationsCache,
contains_arch_indep,
Dominator,
find_live_binary_versions_pass_1,
@@ -1181,7 +1182,9 @@
# version among the input publications in its result.
bpphs = make_bpphs_for_versions(self.factory, ['1.2', '1.1', '1.0'])
make_publications_arch_specific(bpphs, False)
- self.assertEqual(['1.2'], find_live_binary_versions_pass_2(bpphs))
+ cache = ArchSpecificPublicationsCache()
+ self.assertEqual(
+ ['1.2'], find_live_binary_versions_pass_2(bpphs, cache))
def test_find_live_binary_versions_pass_2_blesses_arch_specific(self):
# find_live_binary_versions_pass_2 includes any
@@ -1190,7 +1193,9 @@
versions = list(reversed(['1.%d' % version for version in range(3)]))
bpphs = make_bpphs_for_versions(self.factory, versions)
make_publications_arch_specific(bpphs)
- self.assertEqual(versions, find_live_binary_versions_pass_2(bpphs))
+ cache = ArchSpecificPublicationsCache()
+ self.assertEqual(
+ versions, find_live_binary_versions_pass_2(bpphs, cache))
def test_find_live_binary_versions_pass_2_reprieves_arch_all(self):
# An arch-all BPPH for a BPR built by an SPR that also still has
@@ -1202,8 +1207,9 @@
dependent = self.factory.makeBinaryPackagePublishingHistory(
binarypackagerelease=bpphs[1].binarypackagerelease)
make_publications_arch_specific([dependent], True)
+ cache = ArchSpecificPublicationsCache()
self.assertEqual(
- ['1.2', '1.1'], find_live_binary_versions_pass_2(bpphs))
+ ['1.2', '1.1'], find_live_binary_versions_pass_2(bpphs, cache))
class TestDominationHelpers(TestCaseWithFactory):
@@ -1229,3 +1235,78 @@
def test_contains_arch_indep_says_False_for_empty_list(self):
self.assertFalse(contains_arch_indep([]))
+
+
+class TestArchSpecificPublicationsCache(TestCaseWithFactory):
+ """Tests for `ArchSpecificPublicationsCache`."""
+
+ layer = ZopelessDatabaseLayer
+
+ def makeCache(self):
+ """Shorthand: create a ArchSpecificPublicationsCache."""
+ return ArchSpecificPublicationsCache()
+
+ def makeSPR(self):
+ """Create a `BinaryPackageRelease`."""
+ # Return an un-proxied SPR. This is script code, so it won't be
+ # running into them in real life.
+ return removeSecurityProxy(self.factory.makeSourcePackageRelease())
+
+ def makeBPPH(self, spr=None, arch_specific=True, archive=None,
+ distroseries=None):
+ """Create a `BinaryPackagePublishingHistory`."""
+ if spr is None:
+ spr = self.makeSPR()
+ bpb = self.factory.makeBinaryPackageBuild(source_package_release=spr)
+ bpr = self.factory.makeBinaryPackageRelease(
+ build=bpb, architecturespecific=arch_specific)
+ das = self.factory.makeDistroArchSeries(distroseries=distroseries)
+ return removeSecurityProxy(
+ self.factory.makeBinaryPackagePublishingHistory(
+ binarypackagerelease=bpr, archive=archive,
+ distroarchseries=das, pocket=PackagePublishingPocket.UPDATES,
+ status=PackagePublishingStatus.PUBLISHED))
+
+ def test_getKey_is_consistent_and_distinguishing(self):
+ # getKey consistently returns the same key for the same BPPH,
+ # but different keys for non-matching BPPHs.
+ bpphs = [
+ self.factory.makeBinaryPackagePublishingHistory()
+ for counter in range(2)]
+ cache = self.makeCache()
+ self.assertContentEqual(
+ [cache.getKey(bpph) for bpph in bpphs],
+ set(cache.getKey(bpph) for bpph in bpphs * 2))
+
+ def test_hasArchSpecificPublications_is_consistent_and_correct(self):
+ # hasArchSpecificPublications consistently returns the same
+ # result for the same key; different publications can produce
+ # different results.
+ spr = self.makeSPR()
+ dependent = self.makeBPPH(spr, arch_specific=True)
+ bpph1 = self.makeBPPH(
+ spr, arch_specific=False, archive=dependent.archive,
+ distroseries=dependent.distroseries)
+ bpph2 = self.makeBPPH(arch_specific=False)
+ cache = self.makeCache()
+ self.assertEqual(
+ [True, True, False, False],
+ [
+ cache.hasArchSpecificPublications(bpph1),
+ cache.hasArchSpecificPublications(bpph1),
+ cache.hasArchSpecificPublications(bpph2),
+ cache.hasArchSpecificPublications(bpph2),
+ ])
+
+ def test_hasArchSpecificPublications_caches_results(self):
+ # Results are cached, so once the presence of archive-specific
+ # publications has been looked up in the database, the query is
+ # not performed again for the same inputs.
+ spr = self.makeSPR()
+ self.makeBPPH(spr, arch_specific=True)
+ bpph = self.makeBPPH(spr, arch_specific=False)
+ cache = self.makeCache()
+ cache.hasArchSpecificPublications(bpph)
+ spr.getActiveArchSpecificPublications = FakeMethod()
+ cache.hasArchSpecificPublications(bpph)
+ self.assertEqual(0, spr.getActiveArchSpecificPublications.call_count)
=== modified file 'lib/lp/soyuz/model/publishing.py'
--- lib/lp/soyuz/model/publishing.py 2011-11-03 17:32:41 +0000
+++ lib/lp/soyuz/model/publishing.py 2011-11-04 12:37:24 +0000
@@ -1133,51 +1133,6 @@
section=self.section,
priority=self.priority)
- def getOtherPublicationsForSameSource(self, include_archindep=False):
- """Return all the other published or pending binaries for this
- source.
-
- For example if source package foo builds:
- foo - i386
- foo - amd64
- foo-common - arch-all (published in i386 and amd64)
- then if this publication is the arch-all amd64, return foo(i386),
- foo(amd64). If include_archindep is True then also return
- foo-common (i386)
-
- :param include_archindep: If True, return architecture independent
- publications too. Defaults to False.
-
- :return: an iterable of `BinaryPackagePublishingHistory`
- """
- # Avoid circular wotsits.
- from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
- from lp.soyuz.model.distroarchseries import DistroArchSeries
-
- pubs = [
- BinaryPackageBuild.source_package_release_id ==
- self.binarypackagerelease.build.source_package_release_id,
- BinaryPackageRelease.build == BinaryPackageBuild.id,
- BinaryPackagePublishingHistory.binarypackagereleaseID ==
- BinaryPackageRelease.id,
- BinaryPackagePublishingHistory.archiveID == self.archive.id,
- BinaryPackagePublishingHistory.distroarchseriesID ==
- DistroArchSeries.id,
- DistroArchSeries.distroseriesID == self.distroseries.id,
- BinaryPackagePublishingHistory.pocket == self.pocket,
- BinaryPackagePublishingHistory.status.is_in(
- active_publishing_status),
- BinaryPackagePublishingHistory.id != self.id
- ]
-
- if not include_archindep:
- pubs.append(BinaryPackageRelease.architecturespecific == True)
-
- return IMasterStore(BinaryPackagePublishingHistory).find(
- BinaryPackagePublishingHistory,
- *pubs
- )
-
def supersede(self, dominant=None, logger=None):
"""See `IBinaryPackagePublishingHistory`."""
# At this point only PUBLISHED (ancient versions) or PENDING (
=== modified file 'lib/lp/soyuz/model/sourcepackagerelease.py'
--- lib/lp/soyuz/model/sourcepackagerelease.py 2011-08-30 12:11:41 +0000
+++ lib/lp/soyuz/model/sourcepackagerelease.py 2011-11-04 12:37:24 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
# pylint: disable-msg=E0611,W0212
@@ -10,17 +10,17 @@
]
+import datetime
+import operator
+import re
+from StringIO import StringIO
+
import apt_pkg
-import datetime
from debian.changelog import (
Changelog,
ChangelogCreateError,
ChangelogParseError,
)
-import operator
-import re
-from StringIO import StringIO
-
import pytz
import simplejson
from sqlobject import (
@@ -73,7 +73,10 @@
from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
from lp.soyuz.model.files import SourcePackageReleaseFile
from lp.soyuz.model.packagediff import PackageDiff
-from lp.soyuz.model.publishing import SourcePackagePublishingHistory
+from lp.soyuz.model.publishing import (
+ BinaryPackagePublishingHistory,
+ SourcePackagePublishingHistory,
+ )
from lp.soyuz.model.queue import (
PackageUpload,
PackageUploadSource,
@@ -646,3 +649,46 @@
output = "\n\n".join(chunks)
return output.decode("utf-8", "replace")
+
+ def getActiveArchSpecificPublications(self, archive, distroseries,
+ pocket):
+ """Find architecture-specific binary publications for this release.
+
+ For example, say source package release contains binary packages of:
+ * "foo" for i386 (pending in i386)
+ * "foo" for amd64 (published in amd64)
+ * "foo-common" for the "all" architecture (pending or published in
+ various real processor architectures)
+
+ In that case, this search will return foo(i386) and foo(amd64). The
+ dominator uses this when figuring out whether foo-common can be
+ superseded: we don't track dependency graphs, but we know that the
+ architecture-specific "foo" releases are likely to depend on the
+ architecture-independent foo-common release.
+
+ :param archive: The `Archive` to search.
+ :param distroseries: The `DistroSeries` to search.
+ :param pocket: The `PackagePublishingPocket` to search.
+ :return: A Storm result set of active, architecture-specific
+ `BinaryPackagePublishingHistory` objects for this source package
+ release and the given `archive`, `distroseries`, and `pocket`.
+ """
+ # Avoid circular imports.
+ from lp.soyuz.interfaces.publishing import active_publishing_status
+ from lp.soyuz.model.binarypackagerelease import BinaryPackageRelease
+ from lp.soyuz.model.distroarchseries import DistroArchSeries
+
+ return Store.of(self).find(
+ BinaryPackagePublishingHistory,
+ BinaryPackageBuild.source_package_release_id == self.id,
+ BinaryPackageRelease.build == BinaryPackageBuild.id,
+ BinaryPackagePublishingHistory.binarypackagereleaseID ==
+ BinaryPackageRelease.id,
+ BinaryPackagePublishingHistory.archiveID == archive.id,
+ BinaryPackagePublishingHistory.distroarchseriesID ==
+ DistroArchSeries.id,
+ DistroArchSeries.distroseriesID == distroseries.id,
+ BinaryPackagePublishingHistory.pocket == pocket,
+ BinaryPackagePublishingHistory.status.is_in(
+ active_publishing_status),
+ BinaryPackageRelease.architecturespecific == True)
=== modified file 'lib/lp/soyuz/tests/test_publishing.py'
--- lib/lp/soyuz/tests/test_publishing.py 2011-10-26 02:14:52 +0000
+++ lib/lp/soyuz/tests/test_publishing.py 2011-11-04 12:37:24 +0000
@@ -1505,61 +1505,6 @@
foo_src_pub, foo_bin_pub, foo_one_common_pubs,
foo_two_common_pubs, foo_three_pub)
- def test_getOtherPublicationsForSameSource(self):
- # By default getOtherPublicationsForSameSource should return all
- # of the other binaries built by the same source as the passed
- # binary publication, except the arch-indep ones.
- (foo_src_pub, foo_bin_pub, foo_one_common_pubs, foo_two_common_pubs,
- foo_three_pub) = self._makeMixedSingleBuildPackage()
-
- foo_one_common_pub = foo_one_common_pubs[0]
- others = foo_one_common_pub.getOtherPublicationsForSameSource()
- others = list(others)
-
- self.assertContentEqual([foo_three_pub, foo_bin_pub], others)
-
- def test_getOtherPublicationsForSameSource_include_archindep(self):
- # Check that the arch-indep binaries are returned if requested.
- (foo_src_pub, foo_bin_pub, foo_one_common_pubs, foo_two_common_pubs,
- foo_three_pub) = self._makeMixedSingleBuildPackage()
-
- foo_one_common_pub = foo_one_common_pubs[0]
- others = foo_one_common_pub.getOtherPublicationsForSameSource(
- include_archindep=True)
- others = list(others)
-
- # We expect all publications created above to be returned,
- # except the one we use to call the method on.
- expected = [foo_three_pub, foo_bin_pub]
- expected.extend(foo_one_common_pubs[1:])
- expected.extend(foo_two_common_pubs)
- self.assertContentEqual(expected, others)
-
- def test_getOtherPublicationsForSameSource_inactive(self):
- # Check that inactive publications are not returned.
- (foo_src_pub, foo_bin_pub, foo_one_common_pubs, foo_two_common_pubs,
- foo_three_pub) = self._makeMixedSingleBuildPackage()
- foo_bin_pub.status = PackagePublishingStatus.SUPERSEDED
- foo_three_pub.status = PackagePublishingStatus.SUPERSEDED
- foo_one_common_pub = foo_one_common_pubs[0]
- others = foo_one_common_pub.getOtherPublicationsForSameSource()
- others = list(others)
-
- self.assertEqual(0, len(others))
-
- def test_getOtherPublicationsForSameSource_multiple_versions(self):
- # Check that publications for only the same version as the
- # context binary publication are returned.
- (foo_src_pub, foo_bin_pub, foo_one_common_pubs, foo_two_common_pubs,
- foo_three_pub) = self._makeMixedSingleBuildPackage(version="1.0")
- self._makeMixedSingleBuildPackage(version="1.1")
-
- foo_one_common_pub = foo_one_common_pubs[0]
- others = foo_one_common_pub.getOtherPublicationsForSameSource()
- others = list(others)
-
- self.assertContentEqual([foo_three_pub, foo_bin_pub], others)
-
class TestGetBuiltBinaries(TestNativePublishingBase):
"""Test SourcePackagePublishingHistory.getBuiltBinaries() works."""
=== modified file 'lib/lp/soyuz/tests/test_sourcepackagerelease.py'
--- lib/lp/soyuz/tests/test_sourcepackagerelease.py 2011-09-26 07:33:00 +0000
+++ lib/lp/soyuz/tests/test_sourcepackagerelease.py 2011-11-04 12:37:24 +0000
@@ -9,6 +9,7 @@
import transaction
from zope.component import getUtility
+from zope.security.proxy import removeSecurityProxy
from canonical.testing.layers import (
LaunchpadFunctionalLayer,
@@ -18,7 +19,12 @@
from lp.buildmaster.enums import BuildStatus
from lp.registry.interfaces.pocket import PackagePublishingPocket
from lp.services.tarfile_helpers import LaunchpadWriteTarFile
-from lp.soyuz.enums import SourcePackageFormat
+from lp.soyuz.enums import (
+ ArchivePurpose,
+ PackagePublishingStatus,
+ SourcePackageFormat,
+ )
+from lp.soyuz.interfaces.publishing import active_publishing_status
from lp.soyuz.interfaces.sourcepackageformat import (
ISourcePackageFormatSelectionSet,
)
@@ -105,6 +111,88 @@
self.assertEqual(expected_changelog, observed)
+class TestGetActiveArchSpecificPublications(TestCaseWithFactory):
+
+ layer = ZopelessDatabaseLayer
+
+ def makeSPR(self):
+ """Create a `SourcePackageRelease`."""
+ # Return an un-proxied SPR. This test is for script code; it
+ # won't get proxied objects in real life.
+ return removeSecurityProxy(self.factory.makeSourcePackageRelease())
+
+ def makeBPPHs(self, spr, number=1):
+ """Create `BinaryPackagePublishingHistory` object(s).
+
+ Each of the publications will be active and architecture-specific.
+ Each will be for the same archive, distroseries, and pocket.
+
+ Since the tests need to create a pocket mismatch, it is guaranteed
+ that the BPPHs are for the UPDATES pocket.
+ """
+ bpbs = [
+ self.factory.makeBinaryPackageBuild(source_package_release=spr)
+ for counter in range(number)]
+ bprs = [
+ self.factory.makeBinaryPackageRelease(
+ build=bpb, architecturespecific=True)
+ for bpb in bpbs]
+
+ das = self.factory.makeDistroArchSeries()
+ distroseries = das.distroseries
+ archive = distroseries.main_archive
+ pocket = PackagePublishingPocket.UPDATES
+ return [
+ removeSecurityProxy(
+ self.factory.makeBinaryPackagePublishingHistory(
+ archive=archive, distroarchseries=das, pocket=pocket,
+ binarypackagerelease=bpr,
+ status=PackagePublishingStatus.PUBLISHED))
+ for bpr in bprs]
+
+ def test_getActiveArchSpecificPublications_finds_only_matches(self):
+ spr = self.makeSPR()
+ bpphs = self.makeBPPHs(spr, 5)
+
+ # This BPPH will match our search.
+ match = bpphs[0]
+
+ distroseries = match.distroseries
+ distro = distroseries.distribution
+
+ # These BPPHs will not match our search, each because they fail
+ # one search parameter.
+ bpphs[1].archive = self.factory.makeArchive(
+ distribution=distro, purpose=ArchivePurpose.PARTNER)
+ bpphs[2].distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=self.factory.makeDistroSeries(distribution=distro))
+ bpphs[3].pocket = PackagePublishingPocket.SECURITY
+ bpphs[4].binarypackagerelease.architecturespecific = False
+
+ self.assertContentEqual(
+ [match], spr.getActiveArchSpecificPublications(
+ match.archive, match.distroseries, match.pocket))
+
+ def test_getActiveArchSpecificPublications_detects_absence(self):
+ spr = self.makeSPR()
+ distroseries = spr.upload_distroseries
+ result_set = spr.getActiveArchSpecificPublications(
+ distroseries.main_archive, distroseries,
+ self.factory.getAnyPocket())
+ self.assertFalse(result_set.any())
+
+ def test_getActiveArchSpecificPublications_filters_status(self):
+ spr = self.makeSPR()
+ bpphs = self.makeBPPHs(spr, len(PackagePublishingStatus.items))
+ for bpph, status in zip(bpphs, PackagePublishingStatus.items):
+ bpph.status = status
+ by_status = dict((bpph.status, bpph) for bpph in bpphs)
+ self.assertContentEqual(
+ [by_status[status] for status in active_publishing_status],
+ spr.getActiveArchSpecificPublications(
+ bpphs[0].archive, bpphs[0].distroseries, bpphs[0].pocket))
+
+
class TestSourcePackageReleaseGetBuildByArch(TestCaseWithFactory):
"""Tests for SourcePackageRelease.getBuildByArch()."""
