launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #07907
[Merge] lp:~cjwatson/launchpad/packageset-score into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/packageset-score into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #990219 in Launchpad itself: "Reprioritize package build scores based on packageset"
https://bugs.launchpad.net/launchpad/+bug/990219
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/packageset-score/+merge/105915
== Summary ==
Make it possible to adjust build scores by packagesets, so that we can favour builds that are more likely to be needed to build images. See bug 990219.
== Proposed fix ==
In a previous branch, I added a score column on Packageset. This adds the code to make use of it, and a webservice method restricted to launchpad-buildd-admins to set it.
== Pre-implementation notes ==
I discussed this briefly with William Grant on IRC; the notes are in the bug. I discussed the permission handling with Curtis Hovey.
== Implementation details ==
The main awkwardness was where to put the webservice method to set the score, since nothing else in Packageset is restricted to launchpad-buildd-admins. In the end, at Curtis' suggestion, I created a new interface requiring launchpad.Moderate, and made that be AdminByBuilddAdmins. This might get more complicated in future if Packageset needs to grow new methods with other restrictions, but we can cross that bridge if and when we come to it.
If a package is in multiple package sets, it gets the maximum of their scores, not (say) the sum. The latter seems likely to be overkill.
As usual, I had to spend some time reducing LoC. I did some refactoring of test_buildpackagejob, which helped, but wasn't enough on its own. Eventually I noticed that the buildd-scoring doctest covers substantially the same ground as test_buildpackagejob, so I destroyed it and made sure that everything previously in it is now covered by unit tests.
== Tests ==
bin/test -vvct TestBuildPackageJobScore -t TestBuildQueueManual
== Demo and Q/A ==
Add hello to some packageset on dogfood and set that packageset's score to something non-zero. Upload hello to quantal on dogfood with urgency=low. Its initial score should be the RELEASE/main/low default of 2505 plus the packageset score.
Double-check, for good measure, that a user not in launchpad-buildd-admins can't set a packageset score even if they can upload packages in that packageset. However, they should be able to see the score in the API.
== Lint ==
Just one:
./database/schema/patch-2209-18-0.sql
8: Line exceeds 80 characters.
This is a COMMENT statement, where the convention appears to be to not wrap.
--
https://code.launchpad.net/~cjwatson/launchpad/packageset-score/+merge/105915
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/packageset-score into lp:launchpad.
=== modified file 'database/schema/patch-2209-18-0.sql'
--- database/schema/patch-2209-18-0.sql 2012-05-08 18:55:36 +0000
+++ database/schema/patch-2209-18-0.sql 2012-05-16 01:27:21 +0000
@@ -5,4 +5,6 @@
ALTER TABLE Packageset ADD COLUMN score INTEGER DEFAULT 0 NOT NULL;
+COMMENT ON COLUMN Packageset.score IS 'Build score bonus for packages in this package set.';
+
INSERT INTO LaunchpadDatabaseRevision VALUES (2209, 18, 0);
=== modified file 'lib/lp/buildmaster/tests/test_buildqueue.py'
--- lib/lp/buildmaster/tests/test_buildqueue.py 2012-01-01 02:58:52 +0000
+++ lib/lp/buildmaster/tests/test_buildqueue.py 2012-05-16 01:27:21 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
# pylint: disable-msg=C0324
@@ -1412,3 +1412,19 @@
assign_to_builder(self, 'xxr-daptup', 1, None)
postgres_build, postgres_job = find_job(self, 'postgres', '386')
check_estimate(self, postgres_job, 120)
+
+
+class TestBuildQueueManual(TestCaseWithFactory):
+ layer = ZopelessDatabaseLayer
+
+ def _makeBuildQueue(self):
+ """Produce a `BuildQueue` object to test."""
+ return self.factory.makeSourcePackageRecipeBuildJob()
+
+ def test_manualScore_prevents_rescoring(self):
+ # Manually-set scores are fixed.
+ buildqueue = self._makeBuildQueue()
+ initial_score = buildqueue.lastscore
+ buildqueue.manualScore(initial_score + 5000)
+ buildqueue.score()
+ self.assertEqual(initial_score + 5000, buildqueue.lastscore)
=== modified file 'lib/lp/security.py'
--- lib/lp/security.py 2012-05-14 23:59:33 +0000
+++ lib/lp/security.py 2012-05-16 01:27:21 +0000
@@ -12,7 +12,6 @@
]
from zope.component import (
- getAdapter,
getUtility,
queryAdapter,
)
@@ -2654,6 +2653,11 @@
return user.isOwner(self.obj) or user.in_admin
+class ModeratePackageset(AdminByBuilddAdmin):
+ permission = 'launchpad.Moderate'
+ usedfor = IPackageset
+
+
class EditPackagesetSet(AuthorizationBase):
permission = 'launchpad.Edit'
usedfor = IPackagesetSet
=== modified file 'lib/lp/soyuz/configure.zcml'
--- lib/lp/soyuz/configure.zcml 2012-05-04 13:21:43 +0000
+++ lib/lp/soyuz/configure.zcml 2012-05-16 01:27:21 +0000
@@ -1,4 +1,4 @@
-<!-- Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+<!-- Copyright 2009-2012 Canonical Ltd. This software is licensed under the
GNU Affero General Public License version 3 (see the file LICENSE).
-->
@@ -787,6 +787,9 @@
<require
permission="launchpad.Edit"
interface="lp.soyuz.interfaces.packageset.IPackagesetEdit"/>
+ <require
+ permission="launchpad.Moderate"
+ interface="lp.soyuz.interfaces.packageset.IPackagesetRestricted"/>
</class>
<class
class="lp.soyuz.model.packageset.PackagesetSet">
=== removed file 'lib/lp/soyuz/doc/buildd-scoring.txt'
--- lib/lp/soyuz/doc/buildd-scoring.txt 2012-01-20 15:42:44 +0000
+++ lib/lp/soyuz/doc/buildd-scoring.txt 1970-01-01 00:00:00 +0000
@@ -1,262 +0,0 @@
-Buildd Scoring
-==============
-
-Some tests for build jobs scoring implementation, which envolves the
-analysis of each job pending in the queue. The actions to be performed are
-described in <https://launchpad.canonical.com/AutoBuildManagement>.
-A summary:
-
- * ETA to build (smaller == more points)
- * Time spent in build queue (longer == more points)
- * urgency
- * priority/seed/component (BASE|DESKTOP|SUPPORTED) [PEND]
- * Overarching policy (SECURITY/UPDATES/RELEASE) [PEND]
- * Per-archive score delta.
-
- >>> import datetime
- >>> import pytz
- >>> LOCAL_NOW = datetime.datetime.now(pytz.timezone('UTC'))
-
-Let's create a 'mock' class which emulate the real behaviour of
-BuildQueue entries.
-
- >>> from lp.registry.interfaces.distribution import IDistributionSet
-
- >>> ubuntu = getUtility(IDistributionSet)['ubuntu']
- >>> hoary = ubuntu['hoary']
- >>> hoary386 = hoary['i386']
- >>> hoary386.title
- u'The Hoary Hedgehog Release for i386 (x86)'
-
- >>> from lp.registry.interfaces.pocket import PackagePublishingPocket
- >>> from lp.registry.interfaces.sourcepackage import SourcePackageUrgency
- >>> from lp.soyuz.enums import PackagePublishingStatus
- >>> from lp.soyuz.tests.test_publishing import (
- ... SoyuzTestPublisher)
- >>> from lp.testing.dbuser import (
- ... lp_dbuser,
- ... switch_dbuser,
- ... )
-
- >>> test_publisher = SoyuzTestPublisher()
-
- >>> with lp_dbuser():
- ... test_publisher.prepareBreezyAutotest()
-
- >>> version = 1
-
- >>> def setUpBuildQueueEntry(
- ... component_name='main', urgency=SourcePackageUrgency.HIGH,
- ... pocket=PackagePublishingPocket.RELEASE,
- ... date_created=LOCAL_NOW, manual=False, archive=None):
- ... global version
- ... with lp_dbuser():
- ... pub = test_publisher.getPubSource(
- ... sourcename='test-build', version=str(version),
- ... distroseries=hoary, component=component_name,
- ... urgency=urgency, pocket=pocket,
- ... status=PackagePublishingStatus.PUBLISHED, archive=archive)
- ... version += 1
- ... build = pub.sourcepackagerelease.createBuild(
- ... hoary386, pub.pocket, pub.archive)
- ...
- ... build_queue = build.queueBuild()
- ... from zope.security.proxy import removeSecurityProxy
- ... naked_build_queue = removeSecurityProxy(build_queue)
- ... naked_build_queue.job.date_created = date_created
- ... naked_build_queue.manual = manual
- ...
- ... return build_queue
-
-
- * 1500 points for pocket 'RELEASE',
- * 1000 points for component 'main',
- * 15 points for urgency HIGH.
- * nothing for queue_time
-
- >>> bq0 = setUpBuildQueueEntry()
-
- >>> bq0.score()
- >>> bq0.lastscore
- 2515
-
-If the archive is private, its score is boosted by 10000:
-
- >>> switch_dbuser('launchpad')
- >>> private_ppa = factory.makeArchive()
- >>> private_ppa.buildd_secret = "secret"
- >>> private_ppa.private = True
- >>> bq1 = setUpBuildQueueEntry(archive=private_ppa)
- >>> bq1.score()
- >>> bq1.lastscore
- 12515
-
-The archive can also have a delta applied to all its build scores. Setting
-IArchive.relative_build_score to boost by 100 changes the lastscore value
-appropriately.
-
- >>> private_ppa.relative_build_score = 100
- >>> bq1.score()
- >>> bq1.lastscore
- 12615
-
-The delta can also be negative.
-
- >>> private_ppa.relative_build_score = -100
- >>> bq1.score()
- >>> bq1.lastscore
- 12415
-
- >>> private_ppa.relative_build_score = 0
- >>> switch_dbuser(test_dbuser)
-
-
- * 1500 points for pocket 'RELEASE',
- * 1000 points for main components
- * 5 point for priority LOW
- * nothing for queue_time
-
- >>> time1 = LOCAL_NOW - datetime.timedelta(seconds=290)
- >>> bq1 = setUpBuildQueueEntry(
- ... urgency=SourcePackageUrgency.LOW, date_created=time1)
-
- >>> bq1.score()
- >>> bq1.lastscore
- 2505
-
- * 1500 points for pocket 'RELEASE',
- * 250 points for universe component universe
- * 15 points for priority HIGH
- * 5 points for queue_time ( > 300 seconds)
-
- >>> time2 = LOCAL_NOW - datetime.timedelta(seconds=310)
- >>> bq2 = setUpBuildQueueEntry(
- ... component_name='universe', urgency=SourcePackageUrgency.HIGH,
- ... date_created=time2)
-
- >>> bq2.score()
-
- >>> bq2.lastscore
- 1770
-
- * 1500 points for pocket 'RELEASE',
- * nothing for component multiverse
- * 10 points for MEDIUM priority
- * 10 points for queue_time ( > 900 seconds)
-
- >>> time3 = LOCAL_NOW - datetime.timedelta(seconds=1000)
- >>> bq3 = setUpBuildQueueEntry(
- ... component_name='multiverse', urgency=SourcePackageUrgency.MEDIUM,
- ... date_created=time3)
-
- >>> bq3.score()
-
- >>> bq3.lastscore
- 1520
-
- * 1500 points for pocket 'RELEASE',
- * 1000 points for main component
- * 20 points for EMERGENCY priority
- * 15 points for queue_time ( > 1800 seconds)
-
- >>> time4 = LOCAL_NOW - datetime.timedelta(seconds=1801)
- >>> bq4 = setUpBuildQueueEntry(
- ... component_name='main', urgency=SourcePackageUrgency.EMERGENCY,
- ... date_created=time4)
-
- >>> bq4.score()
- >>> bq4.lastscore
- 2535
-
- * 1500 points for pocket 'RELEASE',
- * 750 points for restricted component
- * 5 points for LOW priority
- * 20 points for queue_time ( > 3600 seconds)
-
- >>> time5 = LOCAL_NOW - datetime.timedelta(seconds=4000)
- >>> bq5 = setUpBuildQueueEntry(
- ... component_name='restricted', urgency=SourcePackageUrgency.LOW,
- ... date_created=time5)
-
- >>> bq5.score()
- >>> bq5.lastscore
- 2275
-
-By setting manual attribute of a BuildQueue entry we prevent it to be
-rescored, which allows us to set an arbitrary value on it.
-
- >>> time6 = LOCAL_NOW
- >>> bq6 = setUpBuildQueueEntry(
- ... urgency=SourcePackageUrgency.LOW, date_created=time6,
- ... manual=True)
-
- >>> bq6.lastscore = 5000
-
- >>> bq6.score()
-
- >>> bq6.lastscore
- 5000
-
-Let's see how the score varies for different publishing pockets.
-
-We will start with the lowest priority pocket: backports.
-
- >>> bq7 = setUpBuildQueueEntry(
- ... pocket=PackagePublishingPocket.BACKPORTS)
- >>> bq7.score()
- >>> bq7.lastscore
- 1015
-
-The score will increase by 3000 for the next ranked pocket: release.
-
- >>> bq8 = setUpBuildQueueEntry(
- ... pocket=PackagePublishingPocket.RELEASE)
- >>> bq8.score()
- >>> bq8.lastscore
- 2515
-
-Going to the next ranked pocket (PROPOSED or UPDATES) there will be a
-score increase of 1500. The reason why PROPOSED and UPDATES have the
-same priority is because sources in both pockets are submitted to the
-same policy and should reach their audience as soon as possible (see
-more information about this decision in bug #372491).
-
- >>> bq9 = setUpBuildQueueEntry(
- ... pocket=PackagePublishingPocket.PROPOSED)
- >>> bq9.score()
- >>> bq9.lastscore
- 4015
-
- >>> bqa = setUpBuildQueueEntry(
- ... pocket=PackagePublishingPocket.UPDATES)
- >>> bqa.score()
- >>> bqa.lastscore
- 4015
-
-Placing the build in the SECURITY pocket will push its score
-up by another 1500.
-
- >>> bqb = setUpBuildQueueEntry(
- ... pocket=PackagePublishingPocket.SECURITY)
- >>> bqb.score()
- >>> bqb.lastscore
- 5515
-
-Builds in COPY archives have a score below zero, so they will only
-be considered when there is nothing else to build. Even language-packs
-and build retries will be built before them.
-
- >>> switch_dbuser('launchpad')
- >>> from lp.soyuz.enums import ArchivePurpose
- >>> from lp.soyuz.interfaces.archive import IArchiveSet
- >>> copy = getUtility(IArchiveSet).new(
- ... owner=ubuntu.owner, purpose=ArchivePurpose.COPY,
- ... name='test-rebuild')
-
- >>> bqc = setUpBuildQueueEntry(archive=copy)
- >>> from lp.soyuz.interfaces.binarypackagebuild import (
- ... IBinaryPackageBuildSet)
- >>> build = getUtility(IBinaryPackageBuildSet).getByQueueEntry(bqc)
- >>> bqc.score()
- >>> bqc.lastscore
- -85
=== modified file 'lib/lp/soyuz/interfaces/packageset.py'
--- lib/lp/soyuz/interfaces/packageset.py 2011-12-24 16:54:44 +0000
+++ lib/lp/soyuz/interfaces/packageset.py 2012-05-16 01:27:21 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
# pylint: disable-msg=E0211,E0213
@@ -25,6 +25,7 @@
export_read_operation,
export_write_operation,
exported,
+ mutator_for,
operation_for_version,
operation_parameters,
operation_returns_collection_of,
@@ -94,6 +95,10 @@
description=_(
'Used internally to link package sets across distro series.'))
+ score = exported(Int(
+ title=_("Build score"), required=True, readonly=True,
+ description=_("Build score bonus for packages in this package set.")))
+
def sourcesIncluded(direct_inclusion=False):
"""Get all source names associated with this package set.
@@ -348,7 +353,19 @@
"""
-class IPackageset(IPackagesetViewOnly, IPackagesetEdit):
+class IPackagesetRestricted(Interface):
+ """A writeable interface for restricted attributes of package sets."""
+ export_as_webservice_entry(publish_web_link=False)
+
+ @mutator_for(IPackagesetViewOnly["score"])
+ @operation_parameters(score=copy_field(IPackagesetViewOnly["score"]))
+ @export_write_operation()
+ @operation_for_version("devel")
+ def setScore(score):
+ """Set the build score bonus for this package set."""
+
+
+class IPackageset(IPackagesetViewOnly, IPackagesetEdit, IPackagesetRestricted):
"""An interface for package sets."""
export_as_webservice_entry(publish_web_link=False)
=== modified file 'lib/lp/soyuz/model/buildpackagejob.py'
--- lib/lp/soyuz/model/buildpackagejob.py 2012-03-16 01:25:51 +0000
+++ lib/lp/soyuz/model/buildpackagejob.py 2012-05-16 01:27:21 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
__metaclass__ = type
@@ -38,7 +38,9 @@
SCORE_BY_POCKET,
SCORE_BY_URGENCY,
)
+from lp.soyuz.interfaces.packageset import IPackagesetSet
from lp.soyuz.model.buildfarmbuildjob import BuildFarmBuildJob
+from lp.soyuz.model.packageset import Packageset
class BuildPackageJob(BuildFarmJobOldDerived, Storm):
@@ -95,18 +97,22 @@
score = 0
# Calculates the urgency-related part of the score.
- urgency = SCORE_BY_URGENCY[
- self.build.source_package_release.urgency]
- score += urgency
+ score += SCORE_BY_URGENCY[self.build.source_package_release.urgency]
# Calculates the pocket-related part of the score.
- score_pocket = SCORE_BY_POCKET[self.build.pocket]
- score += score_pocket
+ score += SCORE_BY_POCKET[self.build.pocket]
# Calculates the component-related part of the score.
score += SCORE_BY_COMPONENT.get(
self.build.current_component.name, 0)
+ # Calculates the package-set-related part of the score.
+ package_sets = getUtility(IPackagesetSet).setsIncludingSource(
+ self.build.source_package_release.name,
+ distroseries=self.build.distro_series)
+ if not package_sets.is_empty():
+ score += package_sets.max(Packageset.score)
+
# Calculates the build queue time component of the score.
right_now = datetime.now(pytz.timezone('UTC'))
eta = right_now - self.job.date_created
=== modified file 'lib/lp/soyuz/model/packageset.py'
--- lib/lp/soyuz/model/packageset.py 2012-01-01 02:58:52 +0000
+++ lib/lp/soyuz/model/packageset.py 2012-05-16 01:27:21 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
__metaclass__ = type
@@ -70,6 +70,8 @@
packagesetgroup_id = Int(name='packagesetgroup', allow_none=False)
packagesetgroup = Reference(packagesetgroup_id, 'PackagesetGroup.id')
+ score = Int(allow_none=False)
+
def add(self, data):
"""See `IPackageset`."""
handlers = (
@@ -327,6 +329,10 @@
Packageset.id != self.id)
return _order_result_set(result_set)
+ def setScore(self, score):
+ """See `IPackageset`."""
+ self.score = score
+
class PackagesetSet:
"""See `IPackagesetSet`."""
=== modified file 'lib/lp/soyuz/tests/test_buildpackagejob.py'
--- lib/lp/soyuz/tests/test_buildpackagejob.py 2012-01-01 02:58:52 +0000
+++ lib/lp/soyuz/tests/test_buildpackagejob.py 2012-05-16 01:27:21 +0000
@@ -1,15 +1,21 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Test BuildQueue features."""
-from datetime import timedelta
+from datetime import (
+ datetime,
+ timedelta,
+ )
+from lazr.restfulclient.errors import Unauthorized
+import pytz
from zope.component import getUtility
from zope.security.proxy import removeSecurityProxy
from lp.buildmaster.enums import BuildStatus
from lp.buildmaster.interfaces.builder import IBuilderSet
+from lp.registry.interfaces.person import IPersonSet
from lp.services.webapp.interfaces import (
DEFAULT_FLAVOR,
IStoreSelector,
@@ -24,7 +30,11 @@
from lp.soyuz.model.binarypackagebuild import BinaryPackageBuild
from lp.soyuz.model.processor import ProcessorFamilySet
from lp.soyuz.tests.test_publishing import SoyuzTestPublisher
-from lp.testing import TestCaseWithFactory
+from lp.testing import (
+ api_url,
+ launchpadlib_for,
+ TestCaseWithFactory,
+ )
from lp.testing.layers import (
DatabaseFunctionalLayer,
LaunchpadZopelessLayer,
@@ -147,60 +157,25 @@
self.non_ppa.require_virtualized = False
self.builds = []
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="firefox",
- status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="cobblers",
- status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="thunderpants",
- status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="apg", status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="vim", status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="gcc", status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="bison", status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="flex", status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
- self.builds.extend(
- self.publisher.getPubSource(
- sourcename="postgres",
- status=PackagePublishingStatus.PUBLISHED,
- archive=self.non_ppa,
- architecturehintlist='any').createMissingBuilds())
+ for sourcename in (
+ "gedit",
+ "firefox",
+ "cobblers",
+ "thunderpants",
+ "apg",
+ "vim",
+ "gcc",
+ "bison",
+ "flex",
+ "postgres",
+ ):
+ self.builds.extend(
+ self.publisher.getPubSource(
+ sourcename=sourcename,
+ status=PackagePublishingStatus.PUBLISHED,
+ archive=self.non_ppa,
+ architecturehintlist='any').createMissingBuilds())
+
# We want the builds to have a lot of variety when it comes to score
# and estimated duration etc. so that the queries under test get
# exercised properly.
@@ -257,9 +232,28 @@
layer = DatabaseFunctionalLayer
+ def makeBuildJob(self, purpose=None, private=False, component="main",
+ urgency="high", pocket="RELEASE", age=None):
+ if purpose is not None or private:
+ archive = self.factory.makeArchive(
+ purpose=purpose, private=private)
+ else:
+ archive = None
+ spph = self.factory.makeSourcePackagePublishingHistory(
+ archive=archive, component=component, urgency=urgency)
+ naked_spph = removeSecurityProxy(spph) # needed for private archives
+ build = self.factory.makeBinaryPackageBuild(
+ source_package_release=naked_spph.sourcepackagerelease,
+ pocket=pocket)
+ job = removeSecurityProxy(build).makeJob()
+ if age is not None:
+ removeSecurityProxy(job).job.date_created = (
+ datetime.now(pytz.timezone("UTC")) - timedelta(seconds=age))
+ return job
+
def test_score_unusual_component(self):
spph = self.factory.makeSourcePackagePublishingHistory(
- component='unusual')
+ component="unusual")
build = self.factory.makeBinaryPackageBuild(
source_package_release=spph.sourcepackagerelease)
build.queueBuild()
@@ -268,31 +262,116 @@
job.score()
def test_main_release_low_score(self):
- spph = self.factory.makeSourcePackagePublishingHistory(
- component='main', urgency='low')
- build = self.factory.makeBinaryPackageBuild(
- source_package_release=spph.sourcepackagerelease,
- pocket='RELEASE')
- job = build.makeJob()
- self.assertEquals(2505, job.score())
+ # 1500 (RELEASE) + 1000 (main) + 5 (low) = 2505.
+ job = self.makeBuildJob(component="main", urgency="low")
+ self.assertEqual(2505, job.score())
def test_copy_archive_main_release_low_score(self):
- copy_archive = self.factory.makeArchive(purpose='COPY')
- spph = self.factory.makeSourcePackagePublishingHistory(
- archive=copy_archive, component='main', urgency='low')
- build = self.factory.makeBinaryPackageBuild(
- source_package_release=spph.sourcepackagerelease,
- pocket='RELEASE')
- job = build.makeJob()
- self.assertEquals(-95, job.score())
+ # 1500 (RELEASE) + 1000 (main) + 5 (low) - 2600 (copy archive) = -95.
+ # With this penalty, even language-packs and build retries will be
+ # built before copy archives.
+ job = self.makeBuildJob(
+ purpose="COPY", component="main", urgency="low")
+ self.assertEqual(-95, job.score())
def test_copy_archive_relative_score_is_applied(self):
- copy_archive = self.factory.makeArchive(purpose='COPY')
- removeSecurityProxy(copy_archive).relative_build_score = 2600
- spph = self.factory.makeSourcePackagePublishingHistory(
- archive=copy_archive, component='main', urgency='low')
- build = self.factory.makeBinaryPackageBuild(
- source_package_release=spph.sourcepackagerelease,
- pocket='RELEASE')
- job = build.makeJob()
- self.assertEquals(2505, job.score())
+ # Per-archive relative build scores are applied, in this case
+ # exactly offsetting the copy-archive penalty.
+ job = self.makeBuildJob(
+ purpose="COPY", component="main", urgency="low")
+ removeSecurityProxy(job.build.archive).relative_build_score = 2600
+ self.assertEqual(2505, job.score())
+
+ def test_archive_negative_relative_score_is_applied(self):
+ # Negative per-archive relative build scores are allowed.
+ job = self.makeBuildJob(component="main", urgency="low")
+ removeSecurityProxy(job.build.archive).relative_build_score = -100
+ self.assertEqual(2405, job.score())
+
+ def test_private_archive_bonus_is_applied(self):
+ # Private archives get a bonus of 10000.
+ job = self.makeBuildJob(private=True, component="main", urgency="high")
+ self.assertEqual(12515, job.score())
+
+ def test_main_release_low_recent_score(self):
+ # Builds created less than five minutes ago get no bonus.
+ job = self.makeBuildJob(component="main", urgency="low", age=290)
+ self.assertEqual(2505, job.score())
+
+ def test_universe_release_high_five_minutes_score(self):
+ # 1500 (RELEASE) + 250 (universe) + 15 (high) + 5 (>300s) = 1770.
+ job = self.makeBuildJob(component="universe", urgency="high", age=310)
+ self.assertEqual(1770, job.score())
+
+ def test_multiverse_release_medium_fifteen_minutes_score(self):
+ # 1500 (RELEASE) + 0 (multiverse) + 10 (medium) + 10 (>900s) = 1520.
+ job = self.makeBuildJob(
+ component="multiverse", urgency="medium", age=1000)
+ self.assertEqual(1520, job.score())
+
+ def test_main_release_emergency_thirty_minutes_score(self):
+ # 1500 (RELEASE) + 1000 (main) + 20 (emergency) + 15 (>1800s) = 2535.
+ job = self.makeBuildJob(
+ component="main", urgency="emergency", age=1801)
+ self.assertEqual(2535, job.score())
+
+ def test_restricted_release_low_one_hour_score(self):
+ # 1500 (RELEASE) + 750 (restricted) + 5 (low) + 20 (>3600s) = 2275.
+ job = self.makeBuildJob(
+ component="restricted", urgency="low", age=4000)
+ self.assertEqual(2275, job.score())
+
+ def test_backports_score(self):
+ # BACKPORTS is the lowest-priority pocket.
+ job = self.makeBuildJob(pocket="BACKPORTS")
+ self.assertEqual(1015, job.score())
+
+ def test_release_score(self):
+ # RELEASE ranks next above BACKPORTS.
+ job = self.makeBuildJob(pocket="RELEASE")
+ self.assertEqual(2515, job.score())
+
+ def test_proposed_updates_score(self):
+ # PROPOSED and UPDATES both rank next above RELEASE. The reason why
+ # PROPOSED and UPDATES have the same priority is because sources in
+ # both pockets are submitted to the same policy and should reach
+ # their audience as soon as possible (see more information about
+ # this decision in bug #372491).
+ proposed_job = self.makeBuildJob(pocket="PROPOSED")
+ self.assertEqual(4015, proposed_job.score())
+ updates_job = self.makeBuildJob(pocket="UPDATES")
+ self.assertEqual(4015, updates_job.score())
+
+ def test_security_updates_score(self):
+ # SECURITY is the top-ranked pocket.
+ job = self.makeBuildJob(pocket="SECURITY")
+ self.assertEqual(5515, job.score())
+
+ def test_score_packageset(self):
+ job = self.makeBuildJob(component="main", urgency="low")
+ packageset = self.factory.makePackageset(
+ distroseries=job.build.distro_series)
+ removeSecurityProxy(packageset).add(
+ [job.build.source_package_release.sourcepackagename])
+ removeSecurityProxy(packageset).score = 100
+ self.assertEqual(2605, job.score())
+
+ def test_score_packageset_forbids_non_buildd_admin(self):
+ # Being the owner of a packageset is not enough to allow changing
+ # its build score, since this affects a site-wide resource.
+ person = self.factory.makePerson()
+ packageset = self.factory.makePackageset(owner=person)
+ lp = launchpadlib_for("testing", person)
+ entry = lp.load(api_url(packageset))
+ entry.score = 100
+ self.assertRaises(Unauthorized, entry.lp_save)
+
+ def test_score_packageset_allows_buildd_admin(self):
+ buildd_admins = getUtility(IPersonSet).getByName(
+ "launchpad-buildd-admins")
+ buildd_admin = self.factory.makePerson(member_of=[buildd_admins])
+ packageset = self.factory.makePackageset()
+ lp = launchpadlib_for("testing", buildd_admin)
+ entry = lp.load(api_url(packageset))
+ entry.score = 100
+ entry.lp_save()
=== modified file 'lib/lp/soyuz/tests/test_doc.py'
--- lib/lp/soyuz/tests/test_doc.py 2012-03-27 13:41:38 +0000
+++ lib/lp/soyuz/tests/test_doc.py 2012-05-16 01:27:21 +0000
@@ -132,11 +132,6 @@
special = {
- 'buildd-scoring.txt': LayeredDocFileSuite(
- '../doc/buildd-scoring.txt',
- setUp=builddmasterSetUp,
- layer=LaunchpadZopelessLayer,
- ),
'package-cache.txt': LayeredDocFileSuite(
'../doc/package-cache.txt',
setUp=statisticianSetUp, tearDown=statisticianTearDown,
Follow ups
