launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #16670
[Merge] lp:~cjwatson/launchpad/livefs into lp:launchpad
Colin Watson has proposed merging lp:~cjwatson/launchpad/livefs into lp:launchpad.
Commit message:
Implement live filesystem building.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #1247461 in Launchpad itself: "Move live filesystem building into Launchpad"
https://bugs.launchpad.net/launchpad/+bug/1247461
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/livefs/+merge/217261
== Summary ==
Implement the guts of the master side of live filesystem building (bug 1247461). There are some more bits to do, in particular a garbo job to make sure we don't keep too many files in the librarian and some more browser code, but we should be able to QA this without those additions, and this diff is quite large enough as it is.
== Pre-implementation notes ==
William recommended writing a separate uploader cron job rather than modifying archiveuploader. I did start on this, but I quickly found that I was copying more code from archiveuploader than I was comfortable with, and that I only needed to add a very small amount of code to UploadProcessor to teach it how to handle livefs uploads without the need for a .changes file. If you'd still like me to write a separate cron job after looking at this implementation, I can go back and do that.
== Implementation details ==
This relies on https://code.launchpad.net/~cjwatson/launchpad/db-livefs/+merge/217206, and includes:
* LiveFS, LiveFSBuild, and LiveFSFile interfaces, models, and webservice exports
* A new Person.createLiveFS method
* Minimal browser navigation code to handle Person:+livefs and Archive:+livefsbuild, just enough for webservice methods to work properly
* A new LiveFSBuildBehaviour
* Failure notification mail handling
* archiveuploader code to handle livefs uploads in "process-upload.py --builds"
== LOC Rationale ==
Ahem, yes. +2849. This is something we've wanted to do for a long time to consolidate the stunted livefs build farm into the main one, which will have overall maintenance benefits for the organisation even if not for Launchpad itself; and it looks likely that this is going to be required as part of RTM preparation for the phone. I'll try to claw things back elsewhere, but I think there's widespread agreement that we need this feature in Launchpad.
== Tests ==
bin/test -vvct livefs
== Demo and Q/A ==
me = lp.load(lp.me)
livefs = me.createLiveFS(owner=lp.me, distroseries=trusty, name="ubuntu-desktop", metadata={"project": "ubuntu"})
livefs.requestBuild(archive=ubuntu.main_archive, distroarchseries=trusty_i386, pocket="Release")
... or something along those lines, and then confirm that the build happens, can be uploaded, and can be retrieved.
--
https://code.launchpad.net/~cjwatson/launchpad/livefs/+merge/217261
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~cjwatson/launchpad/livefs into lp:launchpad.
=== modified file 'lib/lp/_schema_circular_imports.py'
--- lib/lp/_schema_circular_imports.py 2014-04-23 14:24:15 +0000
+++ lib/lp/_schema_circular_imports.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Update the interface schema values due to circular imports.
@@ -196,6 +196,14 @@
)
from lp.soyuz.interfaces.buildrecords import IHasBuildRecords
from lp.soyuz.interfaces.distroarchseries import IDistroArchSeries
+from lp.soyuz.interfaces.livefs import (
+ ILiveFS,
+ ILiveFSView,
+ )
+from lp.soyuz.interfaces.livefsbuild import (
+ ILiveFSBuild,
+ ILiveFSFile,
+ )
from lp.soyuz.interfaces.packageset import (
IPackageset,
IPackagesetSet,
@@ -350,6 +358,10 @@
Reference(schema=IDistroSeries))
patch_plain_parameter_type(IPerson, 'createRecipe', 'daily_build_archive',
IArchive)
+patch_entry_return_type(IPerson, 'createLiveFS', ILiveFS)
+patch_plain_parameter_type(IPerson, 'createLiveFS', 'owner', IPerson)
+patch_plain_parameter_type(IPerson, 'createLiveFS', 'distroseries',
+ IDistroSeries)
patch_plain_parameter_type(IPerson, 'getArchiveSubscriptionURL', 'archive',
IArchive)
patch_collection_return_type(
@@ -576,6 +588,13 @@
# IDistroArchSeries
patch_reference_property(IDistroArchSeries, 'main_archive', IArchive)
+# ILiveFSFile
+patch_reference_property(ILiveFSFile, 'livefsbuild', ILiveFSBuild)
+
+# ILiveFSView
+patch_entry_return_type(ILiveFSView, 'requestBuild', ILiveFSBuild)
+ILiveFSView['builds'].value_type.schema = ILiveFSBuild
+
# IPackageset
patch_collection_return_type(
IPackageset, 'setsIncluded', IPackageset)
=== added file 'lib/lp/archiveuploader/livefsupload.py'
--- lib/lp/archiveuploader/livefsupload.py 1970-01-01 00:00:00 +0000
+++ lib/lp/archiveuploader/livefsupload.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,70 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Process a live filesystem upload."""
+
+__metaclass__ = type
+
+import os
+
+from zope.component import getUtility
+
+from lp.buildmaster.enums import BuildStatus
+from lp.services.librarian.interfaces import ILibraryFileAliasSet
+
+
+class LiveFSUpload:
+ """A live filesystem upload.
+
+ Unlike package uploads, these have no .changes files. We simply attach
+ all the files in the upload directory to the appropriate `ILiveFSBuild`.
+ """
+
+ filename_ending_content_type_map = {
+ ".manifest": "text/plain",
+ ".manifest-remove": "text/plain",
+ ".size": "text/plain",
+ }
+
+ def __init__(self, upload_path, logger):
+ """Create a `LiveFSUpload`.
+
+ :param upload_path: A directory containing files to upload.
+ :param logger: The logger to be used.
+ """
+ self.upload_path = upload_path
+ self.logger = logger
+
+ self.librarian = getUtility(ILibraryFileAliasSet)
+
+ def content_type(self, path):
+ for content_type_map in self.filename_ending_content_type_map.items():
+ ending, content_type = content_type_map
+ if path.endswith(ending):
+ return content_type
+ return "application/octet-stream"
+
+ def process(self, build):
+ """Process this upload, loading it into the database."""
+ self.logger.debug("Beginning processing.")
+
+ for dirpath, _, filenames in os.walk(self.upload_path):
+ if dirpath == self.upload_path:
+ # All relevant files will be in a subdirectory; this is a
+ # simple way to avoid uploading any .distro file that may
+ # exist.
+ continue
+ for livefs_file in sorted(filenames):
+ livefs_path = os.path.join(dirpath, livefs_file)
+ libraryfile = self.librarian.create(
+ livefs_file, os.stat(livefs_path).st_size,
+ open(livefs_path, "rb"),
+ self.content_type(livefs_path),
+ restricted=build.archive.private)
+ build.addFile(libraryfile)
+
+ # The master verifies the status to confirm successful upload.
+ self.logger.debug("Updating %s" % build.title)
+ build.updateStatus(BuildStatus.FULLYBUILT)
+
+ self.logger.debug("Finished upload.")
=== added file 'lib/lp/archiveuploader/tests/test_livefsupload.py'
--- lib/lp/archiveuploader/tests/test_livefsupload.py 1970-01-01 00:00:00 +0000
+++ lib/lp/archiveuploader/tests/test_livefsupload.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,84 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test uploads of LiveFSBuilds."""
+
+__metaclass__ = type
+
+import os
+
+from storm.store import Store
+from zope.component import getUtility
+
+from lp.archiveuploader.livefsupload import LiveFSUpload
+from lp.archiveuploader.tests.test_uploadprocessor import (
+ TestUploadProcessorBase,
+ )
+from lp.archiveuploader.uploadprocessor import (
+ UploadHandler,
+ UploadStatusEnum,
+ )
+from lp.buildmaster.enums import BuildStatus
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.services.features.testing import FeatureFixture
+from lp.services.osutils import write_file
+from lp.soyuz.interfaces.livefs import LIVEFS_FEATURE_FLAG
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuildSet
+from lp.testing import TestCase
+from lp.testing.layers import ZopelessDatabaseLayer
+
+
+class TestLiveFSUpload(TestCase):
+ """Test the `LiveFSUpload` class."""
+
+ layer = ZopelessDatabaseLayer
+
+ def test_content_type(self):
+ uploader = LiveFSUpload(None, None)
+ self.assertEqual(
+ "text/plain", uploader.content_type("ubuntu.manifest"))
+ self.assertEqual(
+ "application/octet-stream",
+ uploader.content_type("ubuntu.squashfs"))
+
+
+class TestLiveFSBuildUploads(TestUploadProcessorBase):
+ """End-to-end tests of LiveFS build uploads."""
+
+ def setUp(self):
+ super(TestLiveFSBuildUploads, self).setUp()
+
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+ self.setupBreezy()
+
+ self.switchToAdmin()
+ self.livefs = self.factory.makeLiveFS()
+ self.build = getUtility(ILiveFSBuildSet).new(
+ requester=self.livefs.owner, livefs=self.livefs,
+ archive=self.factory.makeArchive(
+ distribution=self.ubuntu, owner=self.livefs.owner),
+ distroarchseries=self.breezy["i386"],
+ pocket=PackagePublishingPocket.RELEASE)
+ self.build.updateStatus(BuildStatus.UPLOADING)
+ Store.of(self.build).flush()
+ self.switchToUploader()
+ self.options.context = "buildd"
+
+ self.uploadprocessor = self.getUploadProcessor(
+ self.layer.txn, builds=True)
+
+ def test_sets_build_and_state(self):
+ # The upload processor uploads files and sets the correct status.
+ self.assertFalse(self.build.verifySuccessfulUpload())
+ upload_dir = os.path.join(
+ self.incoming_folder, "test", str(self.build.id), "ubuntu")
+ write_file(os.path.join(upload_dir, "ubuntu.squashfs"), "squashfs")
+ write_file(os.path.join(upload_dir, "ubuntu.manifest"), "manifest")
+ handler = UploadHandler.forProcessor(
+ self.uploadprocessor, self.incoming_folder, "test", self.build)
+ result = handler.processLiveFS(self.log)
+ self.assertEqual(
+ UploadStatusEnum.ACCEPTED, result,
+ "LiveFS upload failed\nGot: %s" % self.log.getLogBuffer())
+ self.assertEqual(BuildStatus.FULLYBUILT, self.build.status)
+ self.assertTrue(self.build.verifySuccessfulUpload())
=== modified file 'lib/lp/archiveuploader/uploadprocessor.py'
--- lib/lp/archiveuploader/uploadprocessor.py 2013-08-01 14:09:45 +0000
+++ lib/lp/archiveuploader/uploadprocessor.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Code for 'processing' 'uploads'. Also see nascentupload.py.
@@ -57,6 +57,7 @@
from zope.component import getUtility
from lp.app.errors import NotFoundError
+from lp.archiveuploader.livefsupload import LiveFSUpload
from lp.archiveuploader.nascentupload import (
EarlyReturnUploadError,
NascentUpload,
@@ -82,6 +83,7 @@
IArchiveSet,
NoSuchPPA,
)
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuild
__all__ = [
@@ -641,6 +643,28 @@
"Unable to find %s with id %d. Skipping." %
(job_type, job_id))
+ def processLiveFS(self, logger=None):
+ """Process a live filesystem upload."""
+ assert ILiveFSBuild.providedBy(self.build)
+ if logger is None:
+ logger = self.processor.log
+ try:
+ logger.info("Processing LiveFS upload %s" % self.upload_path)
+ LiveFSUpload(self.upload_path, logger).process(self.build)
+
+ if self.processor.dry_run:
+ logger.info("Dry run, aborting transaction.")
+ self.processor.ztm.abort()
+ else:
+ logger.info(
+ "Committing the transaction and any mails associated "
+ "with this upload.")
+ self.processor.ztm.commit()
+ return UploadStatusEnum.ACCEPTED
+ except:
+ self.processor.ztm.abort()
+ raise
+
def process(self):
"""Process an upload that is the result of a build.
@@ -660,8 +684,11 @@
# because we want the standard cleanup to occur.
recipe_deleted = (ISourcePackageRecipeBuild.providedBy(self.build)
and self.build.recipe is None)
+ is_livefs = ILiveFSBuild.providedBy(self.build)
if recipe_deleted:
result = UploadStatusEnum.FAILED
+ elif is_livefs:
+ result = self.processLiveFS(logger)
else:
self.processor.log.debug("Build %s found" % self.build.id)
[changes_file] = self.locateChangesFiles()
=== modified file 'lib/lp/buildmaster/enums.py'
--- lib/lp/buildmaster/enums.py 2013-10-31 07:30:54 +0000
+++ lib/lp/buildmaster/enums.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Common build interfaces."""
@@ -149,6 +149,12 @@
Generate translation templates from a bazaar branch.
""")
+ LIVEFSBUILD = DBItem(5, """
+ Live filesystem build
+
+ Build a live filesystem from an archive.
+ """)
+
class BuildQueueStatus(DBEnumeratedType):
"""Build queue status.
=== modified file 'lib/lp/registry/browser/person.py'
--- lib/lp/registry/browser/person.py 2014-03-21 03:34:57 +0000
+++ lib/lp/registry/browser/person.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Person-related view classes."""
@@ -264,6 +264,7 @@
)
from lp.soyuz.interfaces.archivesubscriber import IArchiveSubscriberSet
from lp.soyuz.interfaces.binarypackagebuild import IBinaryPackageBuildSet
+from lp.soyuz.interfaces.livefs import ILiveFSSet
from lp.soyuz.interfaces.publishing import ISourcePackagePublishingHistory
from lp.soyuz.interfaces.sourcepackagerelease import ISourcePackageRelease
@@ -494,6 +495,26 @@
"""Traverse to this person's merge queues."""
return self.context.getMergeQueue(name)
+ @stepto('+livefs')
+ def traverse_livefs(self):
+ """Traverse to this person's live filesystem images."""
+ def get_segments(pillar_name):
+ base = [self.context.name, pillar_name]
+ return itertools.chain(iter(base), iter(self.request.stepstogo))
+
+ pillar_name = self.request.stepstogo.next()
+ livefs = getUtility(ILiveFSSet).traverse(get_segments(pillar_name))
+ if livefs is None:
+ raise NotFoundError
+
+ if livefs.distroseries.distribution.name != pillar_name:
+ # This live filesystem was accessed through one of its
+ # distribution's aliases, so we must redirect to its canonical
+ # URL.
+ return self.redirectSubTree(canonical_url(livefs))
+
+ return livefs
+
class PersonSetNavigation(Navigation):
=== modified file 'lib/lp/registry/interfaces/person.py'
--- lib/lp/registry/interfaces/person.py 2014-03-11 10:37:23 +0000
+++ lib/lp/registry/interfaces/person.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Person interfaces."""
@@ -84,6 +84,7 @@
Bool,
Choice,
Datetime,
+ Dict,
Int,
List,
Object,
@@ -1024,12 +1025,32 @@
def getRecipe(name):
"""Return the person's recipe with the given name."""
+ @call_with(registrant=REQUEST_USER)
+ @operation_parameters(
+ owner=Reference(
+ Interface,
+ title=_("The person who registered this live filesystem image.")),
+ distroseries=Reference(
+ Interface, title=_("The owner of this live filesystem image.")),
+ name=TextLine(
+ title=_("The series for which the image should be built.")),
+ metadata=Dict(
+ title=_(
+ "A dict of data about the image. Entries here will be passed "
+ "to the builder slave."),
+ key_type=TextLine()),
+ )
+ @export_factory_operation(Interface, [])
+ @operation_for_version("devel")
+ def createLiveFS(registrant, owner, distroseries, name, metadata):
+ """Create a `LiveFS` owned by this person."""
+
def getMergeQueue(name):
"""Return the person's merge queue with the given name."""
@call_with(requester=REQUEST_USER)
@export_read_operation()
- @operation_returns_collection_of(Interface) # Really IArchiveSubscriber
+ @operation_returns_collection_of(Interface) # Really IArchiveSubscriber
@operation_for_version('devel')
def getArchiveSubscriptions(requester):
"""Return (private) archives subscription for this person."""
=== modified file 'lib/lp/registry/model/person.py'
--- lib/lp/registry/model/person.py 2014-03-11 11:34:08 +0000
+++ lib/lp/registry/model/person.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Implementation classes for a Person."""
@@ -311,6 +311,7 @@
)
from lp.soyuz.interfaces.archive import IArchiveSet
from lp.soyuz.interfaces.archivesubscriber import IArchiveSubscriberSet
+from lp.soyuz.interfaces.livefs import ILiveFSSet
from lp.soyuz.model.archive import (
Archive,
validate_ppa,
@@ -2951,6 +2952,13 @@
SourcePackageRecipe, SourcePackageRecipe.owner == self,
SourcePackageRecipe.name == name).one()
+ def createLiveFS(self, registrant, owner, distroseries, name, metadata):
+ """See `IPerson`."""
+ livefs = getUtility(ILiveFSSet).new(
+ registrant, owner, distroseries, name, metadata)
+ Store.of(livefs).flush()
+ return livefs
+
def getMergeQueue(self, name):
from lp.code.model.branchmergequeue import BranchMergeQueue
return Store.of(self).find(
=== modified file 'lib/lp/security.py'
--- lib/lp/security.py 2014-03-17 21:50:33 +0000
+++ lib/lp/security.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Security policies for using content objects."""
@@ -196,6 +196,8 @@
IBinaryPackageReleaseDownloadCount,
)
from lp.soyuz.interfaces.distroarchseries import IDistroArchSeries
+from lp.soyuz.interfaces.livefs import ILiveFS
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuild
from lp.soyuz.interfaces.packagecopyjob import IPlainPackageCopyJob
from lp.soyuz.interfaces.packageset import (
IPackageset,
@@ -2872,3 +2874,46 @@
sourcepackagename=self.obj.sourcepackagename,
component=None, strict_component=False)
return reason is None
+
+
+class ViewLiveFS(DelegatedAuthorization):
+ permission = 'launchpad.View'
+ usedfor = ILiveFS
+
+ def __init__(self, obj):
+ super(ViewLiveFS, self).__init__(obj, obj.owner, 'launchpad.View')
+
+
+class EditLiveFS(EditByOwnersOrAdmins):
+ usedfor = ILiveFS
+
+
+class ViewLiveFSBuild(DelegatedAuthorization):
+ permission = 'launchpad.View'
+ usedfor = ILiveFSBuild
+
+ def iter_objects(self):
+ yield self.obj.livefs
+ yield self.obj.archive
+
+
+class EditLiveFSBuild(AdminByBuilddAdmin):
+ permission = 'launchpad.Edit'
+ usedfor = ILiveFSBuild
+
+ def checkAuthenticated(self, user):
+ """Check edit access for live filesystem builds.
+
+ Allow admins, buildd admins, the owner of the live filesystem, and
+ the requester of the live filesystem build.
+ """
+ if user.inTeam(self.obj.requester):
+ return True
+ auth_livefs = EditLiveFS(self.obj.livefs)
+ if auth_livefs.checkAuthenticated(user):
+ return True
+ return super(EditLiveFSBuild, self).checkAuthenticated(user)
+
+
+class AdminLiveFSBuild(AdminByBuilddAdmin):
+ usedfor = ILiveFSBuild
=== modified file 'lib/lp/soyuz/adapters/archivedependencies.py'
--- lib/lp/soyuz/adapters/archivedependencies.py 2013-05-02 00:40:14 +0000
+++ lib/lp/soyuz/adapters/archivedependencies.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Archive dependencies helper function.
@@ -119,9 +119,12 @@
If no ancestry could be found, default to 'universe'.
"""
primary_archive = archive.distribution.main_archive
- ancestries = primary_archive.getPublishedSources(
- name=sourcepackagename,
- distroseries=distroseries, exact_match=True)
+ if sourcepackagename is None:
+ ancestries = []
+ else:
+ ancestries = primary_archive.getPublishedSources(
+ name=sourcepackagename,
+ distroseries=distroseries, exact_match=True)
try:
return ancestries[0].component.name
=== modified file 'lib/lp/soyuz/browser/build.py'
--- lib/lp/soyuz/browser/build.py 2014-02-26 03:05:44 +0000
+++ lib/lp/soyuz/browser/build.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Browser views for builds."""
@@ -84,6 +84,7 @@
IBinaryPackageBuildSet,
IBuildRescoreForm,
)
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuildSet
class BuildUrl:
@@ -149,6 +150,17 @@
except NotFoundError:
return None
+ @stepthrough('+livefsbuild')
+ def traverse_livefsbuild(self, name):
+ try:
+ build_id = int(name)
+ except ValueError:
+ return None
+ try:
+ return getUtility(ILiveFSBuildSet).getByID(build_id)
+ except NotFoundError:
+ return None
+
class BuildContextMenu(ContextMenu):
"""Overview menu for build records """
=== modified file 'lib/lp/soyuz/browser/configure.zcml'
--- lib/lp/soyuz/browser/configure.zcml 2014-04-24 07:30:36 +0000
+++ lib/lp/soyuz/browser/configure.zcml 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-<!-- Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+<!-- Copyright 2009-2014 Canonical Ltd. This software is licensed under the
GNU Affero General Public License version 3 (see the file LICENSE).
-->
@@ -728,5 +728,19 @@
template="../templates/packagerelationship-list.pt"
/>
</browser:pages>
+ <browser:url
+ for="lp.soyuz.interfaces.livefs.ILiveFS"
+ path_expression="string:+livefs/${distroseries/distribution/name}/${distroseries/name}/${name}"
+ attribute_to_parent="owner"
+ />
+ <browser:url
+ for="lp.soyuz.interfaces.livefsbuild.ILiveFSBuild"
+ path_expression="string:+livefsbuild/${id}"
+ attribute_to_parent="archive"
+ />
+ <browser:navigation
+ module="lp.soyuz.browser.livefsbuild"
+ classes="LiveFSBuildNavigation"
+ />
</facet>
</configure>
=== added file 'lib/lp/soyuz/browser/livefsbuild.py'
--- lib/lp/soyuz/browser/livefsbuild.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/browser/livefsbuild.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,15 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+__metaclass__ = type
+__all__ = [
+ 'LiveFSBuildNavigation',
+ ]
+
+from lp.services.librarian.browser import FileNavigationMixin
+from lp.services.webapp import Navigation
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuild
+
+
+class LiveFSBuildNavigation(Navigation, FileNavigationMixin):
+ usedfor = ILiveFSBuild
=== added file 'lib/lp/soyuz/browser/tests/test_livefs.py'
--- lib/lp/soyuz/browser/tests/test_livefs.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/browser/tests/test_livefs.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,29 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test live filesystem navigation."""
+
+__metaclass__ = type
+
+from lp.services.features.testing import FeatureFixture
+from lp.soyuz.interfaces.livefs import LIVEFS_FEATURE_FLAG
+from lp.testing import TestCaseWithFactory
+from lp.testing.layers import DatabaseFunctionalLayer
+from lp.testing.publication import test_traverse
+
+
+class TestLiveFSNavigation(TestCaseWithFactory):
+
+ layer = DatabaseFunctionalLayer
+
+ def setUp(self):
+ super(TestLiveFSNavigation, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+
+ def test_livefs(self):
+ livefs = self.factory.makeLiveFS()
+ obj, _, _ = test_traverse(
+ "http://api.launchpad.dev/devel/~%s/+livefs/%s/%s/%s"; % (
+ livefs.owner.name, livefs.distroseries.distribution.name,
+ livefs.distroseries.name, livefs.name))
+ self.assertEqual(livefs, obj)
=== modified file 'lib/lp/soyuz/configure.zcml'
--- lib/lp/soyuz/configure.zcml 2014-02-18 11:40:52 +0000
+++ lib/lp/soyuz/configure.zcml 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-<!-- Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+<!-- Copyright 2009-2014 Canonical Ltd. This software is licensed under the
GNU Affero General Public License version 3 (see the file LICENSE).
-->
@@ -975,6 +975,67 @@
<allow
interface=".interfaces.packagetranslationsuploadjob.IPackageTranslationsUploadJob" />
</class>
+
+ <!-- LiveFS -->
+ <class class=".model.livefs.LiveFS">
+ <require
+ permission="launchpad.View"
+ interface=".interfaces.livefs.ILiveFSView
+ .interfaces.livefs.ILiveFSEditableAttributes"/>
+ <require
+ permission="launchpad.Edit"
+ set_schema=".interfaces.livefs.ILiveFSEditableAttributes"/>
+ </class>
+
+ <!-- LiveFSSet -->
+ <securedutility
+ class=".model.livefs.LiveFSSet"
+ provides=".interfaces.livefs.ILiveFSSet">
+ <allow interface=".interfaces.livefs.ILiveFSSet"/>
+ </securedutility>
+
+ <!-- LiveFSBuild -->
+ <class class=".model.livefsbuild.LiveFSBuild">
+ <require
+ permission="launchpad.View"
+ interface=".interfaces.livefsbuild.ILiveFSBuildView"/>
+ <require
+ permission="launchpad.Edit"
+ interface=".interfaces.livefsbuild.ILiveFSBuildEdit"/>
+ <require
+ permission="launchpad.Admin"
+ interface=".interfaces.livefsbuild.ILiveFSBuildAdmin"/>
+ </class>
+
+ <!-- LiveFSBuildSet -->
+ <securedutility
+ class=".model.livefsbuild.LiveFSBuildSet"
+ provides=".interfaces.livefsbuild.ILiveFSBuildSet">
+ <allow interface=".interfaces.livefsbuild.ILiveFSBuildSet"/>
+ </securedutility>
+ <securedutility
+ class=".model.livefsbuild.LiveFSBuildSet"
+ provides="lp.buildmaster.interfaces.buildfarmjob.ISpecificBuildFarmJobSource"
+ name="LIVEFSBUILD">
+ <allow interface="lp.buildmaster.interfaces.buildfarmjob.ISpecificBuildFarmJobSource"/>
+ </securedutility>
+
+ <!-- LiveFSBuildBehaviour -->
+ <adapter
+ for=".interfaces.livefsbuild.ILiveFSBuild"
+ provides="lp.buildmaster.interfaces.buildfarmjobbehaviour.IBuildFarmJobBehaviour"
+ factory=".model.livefsbuildbehaviour.LiveFSBuildBehaviour"
+ permission="zope.Public"/>
+
+ <!-- LiveFSFile -->
+ <class class=".model.livefsbuild.LiveFSFile">
+ <allow
+ interface=".interfaces.livefsbuild.ILiveFSFile"/>
+ <require
+ permission="launchpad.Edit"
+ set_schema=".interfaces.livefsbuild.ILiveFSFile"/>
+ </class>
+
<webservice:register module="lp.soyuz.interfaces.webservice" />
</configure>
=== added file 'lib/lp/soyuz/emailtemplates/livefsbuild-notification.txt'
--- lib/lp/soyuz/emailtemplates/livefsbuild-notification.txt 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/emailtemplates/livefsbuild-notification.txt 2014-04-29 17:27:20 +0000
@@ -0,0 +1,11 @@
+ * Live Filesystem: %(livefs_name)s
+ * Version: %(version)s
+ * Archive: %(archive_tag)s
+ * Distroseries: %(distroseries)s
+ * Architecture: %(architecturetag)s
+ * Pocket: %(pocket)s
+ * State: %(build_state)s
+ * Duration: %(build_duration)s
+ * Build Log: %(log_url)s
+ * Upload Log: %(upload_log_url)s
+ * Builder: %(builder_url)s
=== added file 'lib/lp/soyuz/interfaces/livefs.py'
--- lib/lp/soyuz/interfaces/livefs.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/interfaces/livefs.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,224 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Live filesystem interfaces."""
+
+__metaclass__ = type
+
+__all__ = [
+ 'ILiveFS',
+ 'ILiveFSEditableAttributes',
+ 'ILiveFSSet',
+ 'ILiveFSView',
+ 'InvalidLiveFSNamespace',
+ 'LIVEFS_FEATURE_FLAG',
+ 'LiveFSBuildAlreadyPending',
+ 'LiveFSFeatureDisabled',
+ 'NoSuchLiveFS',
+ ]
+
+import httplib
+
+from lazr.lifecycle.snapshot import doNotSnapshot
+from lazr.restful.declarations import (
+ call_with,
+ error_status,
+ export_as_webservice_entry,
+ export_factory_operation,
+ export_write_operation,
+ exported,
+ operation_for_version,
+ operation_parameters,
+ REQUEST_USER,
+ )
+from lazr.restful.fields import (
+ CollectionField,
+ Reference,
+ )
+from zope.interface import Interface
+from zope.schema import (
+ Choice,
+ Datetime,
+ Dict,
+ Int,
+ TextLine,
+ )
+from zope.security.interfaces import Unauthorized
+
+from lp import _
+from lp.app.errors import NameLookupFailed
+from lp.app.validators.name import name_validator
+from lp.registry.interfaces.distroseries import IDistroSeries
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.registry.interfaces.role import IHasOwner
+from lp.services.fields import (
+ PersonChoice,
+ PublicPersonChoice,
+ )
+from lp.soyuz.interfaces.archive import IArchive
+from lp.soyuz.interfaces.distroarchseries import IDistroArchSeries
+
+
+LIVEFS_FEATURE_FLAG = u"soyuz.livefs.allow_new"
+
+
+@error_status(httplib.BAD_REQUEST)
+class LiveFSBuildAlreadyPending(Exception):
+ """A build was requested when an identical build was already pending."""
+
+ def __init__(self):
+ super(LiveFSBuildAlreadyPending, self).__init__(
+ "An identical build of this live filesystem image is already "
+ "pending.")
+
+
+@error_status(httplib.UNAUTHORIZED)
+class LiveFSFeatureDisabled(Unauthorized):
+ """Only certain users can create new LiveFS-related objects."""
+
+ def __init__(self):
+ super(LiveFSFeatureDisabled, self).__init__(
+ "You do not have permission to create new live filesystems or "
+ "new live filesystem builds.")
+
+
+class InvalidLiveFSNamespace(Exception):
+ """Raised when someone tries to lookup a namespace with a bad name.
+
+ By 'bad', we mean that the name is unparsable. It might be too short,
+ too long, or malformed in some other way.
+ """
+
+ def __init__(self, name):
+ self.name = name
+ super(InvalidLiveFSNamespace, self).__init__(
+ "Cannot understand namespace name: '%s'" % name)
+
+
+class NoSuchLiveFS(NameLookupFailed):
+ """Raised when we try to load a live filesystem that does not exist."""
+
+ _message_prefix = "No such live filesystem"
+
+
+class ILiveFSView(Interface):
+ """`ILiveFS` attributes that require launchpad.View permission."""
+
+ id = exported(Int(title=_("ID"), required=True, readonly=True))
+
+ date_created = exported(Datetime(
+ title=_("Date created"), required=True, readonly=True))
+
+ registrant = exported(PublicPersonChoice(
+ title=_("Registrant"), required=True, readonly=True,
+ vocabulary="ValidPersonOrTeam",
+ description=_(
+ "The person who registered this live filesystem image.")))
+
+ @call_with(requester=REQUEST_USER)
+ @operation_parameters(
+ archive=Reference(schema=IArchive),
+ distroarchseries=Reference(schema=IDistroArchSeries),
+ pocket=Choice(vocabulary=PackagePublishingPocket),
+ unique_key=TextLine(
+ title=_("A unique key for this build, if required."),
+ required=False),
+ metadata_override=Dict(
+ title=_("A JSON string with a dict of data about the image."),
+ key_type=TextLine(), required=False))
+ # Really ILiveFSBuild, patched in _schema_circular_imports.py.
+ @export_factory_operation(Interface, [])
+ @export_write_operation()
+ @operation_for_version("devel")
+ def requestBuild(requester, archive, distroarchseries, pocket,
+ unique_key=None, metadata_override=None):
+ """Request that the live filesystem be built.
+
+ :param requester: The person requesting the build.
+ :param archive: The IArchive to associate the build with.
+ :param distroarchseries: The architecture to build for.
+ :param pocket: The pocket that should be targeted.
+ :param unique_key: An optional unique key for this build; if set,
+ this identifies a class of builds for this live filesystem.
+ :param metadata_override: An optional JSON string with a dict of
+ data about the image; this will be merged into the metadata dict
+ for the live filesystem.
+ :return: `ILiveFSBuild`.
+ """
+
+ builds = exported(doNotSnapshot(CollectionField(
+ title=_("All builds of this live filesystem."),
+ description=_(
+ "All builds of this live filesystem, sorted in descending order "
+ "of finishing (or starting if not completed successfully)."),
+ # Really ILiveFSBuild, patched in _schema_circular_imports.py.
+ value_type=Reference(schema=Interface), readonly=True)))
+
+
+class ILiveFSEditableAttributes(IHasOwner):
+ """`ILiveFS` attributes that can be edited.
+
+ These attributes need launchpad.View to see, and launchpad.Edit to change.
+ """
+ owner = exported(PersonChoice(
+ title=_("Owner"), required=True, readonly=False,
+ vocabulary="AllUserTeamsParticipationPlusSelf",
+ description=_("The owner of this live filesystem image.")))
+
+ distroseries = exported(Reference(
+ IDistroSeries, title=_("Distro Series"), required=True, readonly=False,
+ description=_("The series for which the image should be built.")))
+
+ name = exported(TextLine(
+ title=_("Name"), required=True, readonly=False,
+ constraint=name_validator,
+ description=_("The name of the live filesystem image.")))
+
+ metadata = exported(Dict(
+ title=_(
+ "A dict of data about the image. Entries here will be passed to "
+ "the builder slave."),
+ key_type=TextLine(), required=True, readonly=False))
+
+
+class ILiveFS(ILiveFSView, ILiveFSEditableAttributes):
+ """A buildable live filesystem image."""
+
+ export_as_webservice_entry(
+ singular_name="livefs", plural_name="livefses", as_of="devel")
+
+
+class ILiveFSSet(Interface):
+ """A utility to create and access live filesystems."""
+
+ def new(registrant, owner, distroseries, name, metadata,
+ date_created=None):
+ """Create an `ILiveFS`."""
+
+ def exists(owner, distroseries, name):
+ """Check to see if a matching live filesystem exists."""
+
+ def get(owner, distroseries, name):
+ """Return the appropriate `ILiveFS` for the given objects."""
+
+ def traverse(segments):
+ """Look up the `ILiveFS` at the path given by 'segments'.
+
+ The iterable 'segments' will be consumed until a live filesystem is
+ found. As soon as a live filesystem is found, it will be returned
+ and the consumption of segments will stop. Thus, there will often
+ be unconsumed segments that can be used for further traversal.
+
+ :param segments: An iterable of names of Launchpad components.
+ The first segment is the username, *not* preceded by a '~'.
+ :raise InvalidNamespace: if there are not enough segments to define
+ a live filesystem.
+ :raise NoSuchPerson: if the person referred to cannot be found.
+ :raise NoSuchDistribution: if the distribution referred to cannot be
+ found.
+ :raise NoSuchDistroSeries: if the distroseries referred to cannot be
+ found.
+ :raise NoSuchLiveFS: if the live filesystem referred to cannot be
+ found.
+ :return: `ILiveFS`.
+ """
=== added file 'lib/lp/soyuz/interfaces/livefsbuild.py'
--- lib/lp/soyuz/interfaces/livefsbuild.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/interfaces/livefsbuild.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,201 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Live filesystem build interfaces."""
+
+__metaclass__ = type
+
+__all__ = [
+ 'ILiveFSBuild',
+ 'ILiveFSBuildSet',
+ 'ILiveFSFile',
+ ]
+
+from lazr.restful.declarations import (
+ export_as_webservice_entry,
+ export_read_operation,
+ export_write_operation,
+ exported,
+ operation_for_version,
+ operation_parameters,
+ )
+from lazr.restful.fields import Reference
+from zope.interface import Interface
+from zope.schema import (
+ Bool,
+ Choice,
+ Dict,
+ Int,
+ TextLine,
+ )
+
+from lp import _
+from lp.buildmaster.interfaces.buildfarmjob import ISpecificBuildFarmJobSource
+from lp.buildmaster.interfaces.packagebuild import IPackageBuild
+from lp.registry.interfaces.person import IPerson
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.services.database.constants import DEFAULT
+from lp.services.librarian.interfaces import ILibraryFileAlias
+from lp.soyuz.interfaces.archive import IArchive
+from lp.soyuz.interfaces.distroarchseries import IDistroArchSeries
+from lp.soyuz.interfaces.livefs import ILiveFS
+
+
+class ILiveFSFile(Interface):
+ """A file produced by a live filesystem build."""
+
+ livefsbuild = Reference(
+ # Really ILiveFSBuild, patched in _schema_circular_imports.py.
+ Interface,
+ title=_("The live filesystem build producing this file."),
+ required=True, readonly=True)
+ libraryfile = Reference(
+ ILibraryFileAlias, title=_("The library file alias for this file."),
+ required=True, readonly=True)
+
+
+class ILiveFSBuildView(IPackageBuild):
+ """`ILiveFSBuild` attributes that require launchpad.View permission."""
+
+ requester = exported(Reference(
+ IPerson,
+ title=_("The person who requested this build."),
+ required=True, readonly=True))
+
+ livefs = exported(Reference(
+ ILiveFS,
+ title=_("The live filesystem to build."),
+ required=True, readonly=True))
+
+ archive = exported(Reference(
+ IArchive,
+ title=_("The archive from which to build the live filesystem."),
+ required=True, readonly=True))
+
+ distroarchseries = exported(Reference(
+ IDistroArchSeries,
+ title=_("The series and architecture for which to build."),
+ required=True, readonly=True))
+
+ pocket = exported(Choice(
+ title=_("The pocket for which to build."),
+ vocabulary=PackagePublishingPocket, required=True, readonly=True))
+
+ unique_key = exported(TextLine(
+ title=_(
+ "An optional unique key; if set, this identifies a class of "
+ "builds for this live filesystem."),
+ required=False, readonly=True))
+
+ metadata_override = exported(Dict(
+ title=_(
+ "A dict of data about the image; this will be merged into the "
+ "metadata dict for the live filesystem."),
+ key_type=TextLine(), required=False, readonly=True))
+
+ is_virtualized = Bool(
+ title=_("If True, this build is virtualized."), readonly=True)
+
+ version = exported(TextLine(
+ title=_("A timestamp-based version identifying this build."),
+ required=True, readonly=True))
+
+ score = exported(Int(
+ title=_("Score of the related build farm job (if any)."),
+ required=False, readonly=True))
+
+ can_be_rescored = exported(Bool(
+ title=_("Can be rescored"),
+ required=True, readonly=True,
+ description=_("Whether this build record can be rescored manually.")))
+
+ can_be_retried = exported(Bool(
+ title=_("Can be retried"),
+ required=True, readonly=True,
+ description=_("Whether this build record can be retried.")))
+
+ can_be_cancelled = exported(Bool(
+ title=_("Can be cancelled"),
+ required=True, readonly=True,
+ description=_("Whether this build record can be cancelled.")))
+
+ def getFiles():
+ """Retrieve the build's `ILiveFSFile` records.
+
+ :return: A result set of (`ILiveFSFile`, `ILibraryFileAlias`,
+ `ILibraryFileContent`).
+ """
+
+ def getFileByName(filename):
+ """Return the corresponding `ILibraryFileAlias` in this context.
+
+ The following file types (and extension) can be looked up:
+
+ * Build log: '.txt.gz'
+ * Upload log: '_log.txt'
+
+ Any filename not matching one of these extensions is looked up as a
+ live filesystem output file.
+
+ :param filename: The filename to look up.
+ :raises NotFoundError: if no file exists with the given name.
+ :return: The corresponding `ILibraryFileAlias`.
+ """
+
+ @export_read_operation()
+ @operation_for_version("devel")
+ def getFileUrls():
+ """URLs for all the files produced by this build.
+
+ :return: A collection of URLs for this build."""
+
+
+class ILiveFSBuildEdit(Interface):
+ """`ILiveFSBuild` attributes that require launchpad.Edit."""
+
+ def addFile(lfa):
+ """Add a file to this build.
+
+ :param lfa: An `ILibraryFileAlias`.
+ :return: An `ILiveFSFile`.
+ """
+
+ @export_write_operation()
+ @operation_for_version("devel")
+ def cancel():
+ """Cancel the build if it is either pending or in progress.
+
+ Call the can_be_cancelled() method prior to this one to find out if
+ cancelling the build is possible.
+
+ If the build is in progress, it is marked as CANCELLING until the
+ buildd manager terminates the build and marks it CANCELLED. If the
+ build is not in progress, it is marked CANCELLED immediately and is
+ removed from the build queue.
+
+ If the build is not in a cancellable state, this method is a no-op.
+ """
+
+
+class ILiveFSBuildAdmin(Interface):
+ """`ILiveFSBuild` attributes that require launchpad.Admin."""
+
+ @operation_parameters(score=Int(title=_("Score"), required=True))
+ @export_write_operation()
+ @operation_for_version("devel")
+ def rescore(score):
+ """Change the build's score."""
+
+
+class ILiveFSBuild(ILiveFSBuildView, ILiveFSBuildEdit, ILiveFSBuildAdmin):
+ """Build information for live filesystem builds."""
+
+ export_as_webservice_entry(singular_name="livefs_build", as_of="devel")
+
+
+class ILiveFSBuildSet(ISpecificBuildFarmJobSource):
+ """Utility for `ILiveFSBuild`."""
+
+ def new(requester, livefs, archive, distroarchseries, pocket,
+ unique_key=None, metadata_override=None, date_created=DEFAULT):
+ """Create an `ILiveFSBuild`."""
=== modified file 'lib/lp/soyuz/interfaces/webservice.py'
--- lib/lp/soyuz/interfaces/webservice.py 2013-09-13 07:09:55 +0000
+++ lib/lp/soyuz/interfaces/webservice.py 2014-04-29 17:27:20 +0000
@@ -1,4 +1,4 @@
-# Copyright 2010-2011 Canonical Ltd. This software is licensed under the
+# Copyright 2010-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""All the interfaces that are exposed through the webservice.
@@ -86,6 +86,8 @@
)
from lp.soyuz.interfaces.buildrecords import IncompatibleArguments
from lp.soyuz.interfaces.distroarchseries import IDistroArchSeries
+from lp.soyuz.interfaces.livefs import ILiveFS
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuild
from lp.soyuz.interfaces.packageset import (
DuplicatePackagesetName,
IPackageset,
=== added file 'lib/lp/soyuz/mail/livefsbuild.py'
--- lib/lp/soyuz/mail/livefsbuild.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/mail/livefsbuild.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,89 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+__metaclass__ = type
+__all__ = [
+ 'LiveFSBuildMailer',
+ ]
+
+from lp.app.browser.tales import DurationFormatterAPI
+from lp.archivepublisher.utils import get_ppa_reference
+from lp.services.config import config
+from lp.services.mail.basemailer import (
+ BaseMailer,
+ RecipientReason,
+ )
+from lp.services.webapp import canonical_url
+
+
+class LiveFSBuildMailer(BaseMailer):
+
+ app = 'soyuz'
+
+ @classmethod
+ def forStatus(cls, build):
+ """Create a mailer for notifying about live filesystem build status.
+
+ :param build: The relevant build.
+ """
+ requester = build.requester
+ recipients = {requester: RecipientReason.forBuildRequester(requester)}
+ return cls(
+ "[LiveFS build #%(build_id)d] %(build_title)s",
+ "livefsbuild-notification.txt", recipients,
+ config.canonical.noreply_from_address, build)
+
+ def __init__(self, subject, template_name, recipients, from_address,
+ build):
+ super(LiveFSBuildMailer, self).__init__(
+ subject, template_name, recipients, from_address,
+ notification_type="livefs-build-status")
+ self.build = build
+
+ def _getHeaders(self, email):
+ """See `BaseMailer`."""
+ headers = super(LiveFSBuildMailer, self)._getHeaders(email)
+ headers["X-Launchpad-Build-State"] = self.build.status.name
+ return headers
+
+ def _getTemplateParams(self, email, recipient):
+ """See `BaseMailer`."""
+ build = self.build
+ params = super(LiveFSBuildMailer, self)._getTemplateParams(
+ email, recipient)
+ params.update({
+ "build_id": build.id,
+ "build_title": build.title,
+ "livefs_name": build.livefs.name,
+ "version": build.version,
+ "distroseries": build.livefs.distroseries,
+ "architecturetag": build.distroarchseries.architecturetag,
+ "pocket": build.pocket.name,
+ "build_state": build.status.title,
+ "build_duration": "",
+ "log_url": "",
+ "upload_log_url": "",
+ "builder_url": "",
+ "build_url": canonical_url(self.build),
+ })
+ if build.archive.is_ppa:
+ archive_tag = "%s PPA" % get_ppa_reference(build.archive)
+ else:
+ archive_tag = "%s primary archive" % (
+ build.archive.distribution.name)
+ params["archive_tag"] = archive_tag
+ if build.duration is not None:
+ duration_formatter = DurationFormatterAPI(build.duration)
+ params["build_duration"] = duration_formatter.approximateduration()
+ if build.log is not None:
+ params["log_url"] = build.log_url
+ if build.upload_log is not None:
+ params["upload_log_url"] = build.upload_log_url
+ if build.builder is not None:
+ params["builder_url"] = canonical_url(build.builder)
+ return params
+
+ def _getFooter(self, params):
+ """See `BaseMailer`."""
+ return ("%(build_url)s\n"
+ "%(reason)s\n" % params)
=== added file 'lib/lp/soyuz/model/livefs.py'
--- lib/lp/soyuz/model/livefs.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/model/livefs.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,210 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+__metaclass__ = type
+__all__ = [
+ 'LiveFS',
+ ]
+
+import pytz
+from storm.locals import (
+ DateTime,
+ Desc,
+ Int,
+ JSON,
+ Reference,
+ Store,
+ Storm,
+ Unicode,
+ )
+from zope.component import getUtility
+from zope.interface import implements
+
+from lp.buildmaster.enums import BuildStatus
+from lp.registry.errors import NoSuchDistroSeries
+from lp.registry.interfaces.distribution import (
+ IDistributionSet,
+ NoSuchDistribution,
+ )
+from lp.registry.interfaces.distroseries import IDistroSeriesSet
+from lp.registry.interfaces.person import (
+ IPersonSet,
+ NoSuchPerson,
+ )
+from lp.registry.interfaces.role import IHasOwner
+from lp.services.database.constants import DEFAULT
+from lp.services.database.interfaces import (
+ IMasterStore,
+ IStore,
+ )
+from lp.services.database.stormexpr import Greatest
+from lp.services.features import getFeatureFlag
+from lp.soyuz.interfaces.livefs import (
+ ILiveFS,
+ ILiveFSSet,
+ InvalidLiveFSNamespace,
+ LIVEFS_FEATURE_FLAG,
+ LiveFSBuildAlreadyPending,
+ LiveFSFeatureDisabled,
+ NoSuchLiveFS,
+ )
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuildSet
+from lp.soyuz.model.archive import Archive
+from lp.soyuz.model.livefsbuild import LiveFSBuild
+
+
+class LiveFS(Storm):
+ """See `ILiveFS`."""
+
+ __storm_table__ = 'LiveFS'
+
+ def __str__(self):
+ return '%s %s' % (self.distroseries, self.name)
+
+ implements(ILiveFS, IHasOwner)
+
+ id = Int(primary=True)
+
+ date_created = DateTime(
+ name='date_created', tzinfo=pytz.UTC, allow_none=False)
+
+ registrant_id = Int(name='registrant', allow_none=False)
+ registrant = Reference(registrant_id, 'Person.id')
+
+ owner_id = Int(name='owner', allow_none=False)
+ owner = Reference(owner_id, 'Person.id')
+
+ distroseries_id = Int(name='distroseries', allow_none=False)
+ distroseries = Reference(distroseries_id, 'DistroSeries.id')
+
+ name = Unicode(name='name', allow_none=False)
+
+ metadata = JSON('json_data')
+
+ def __init__(self, registrant, owner, distroseries, name, metadata,
+ date_created):
+ """Construct a `LiveFS`."""
+ if not getFeatureFlag(LIVEFS_FEATURE_FLAG):
+ raise LiveFSFeatureDisabled
+ super(LiveFS, self).__init__()
+ self.registrant = registrant
+ self.owner = owner
+ self.distroseries = distroseries
+ self.name = name
+ self.metadata = metadata
+ self.date_created = date_created
+
+ def requestBuild(self, requester, archive, distroarchseries, pocket,
+ unique_key=None, metadata_override=None):
+ """See `ILiveFS`."""
+ pending = IStore(self).find(
+ LiveFSBuild,
+ LiveFSBuild.livefs_id == self.id,
+ LiveFSBuild.archive_id == archive.id,
+ LiveFSBuild.distroarchseries_id == distroarchseries.id,
+ LiveFSBuild.pocket == pocket,
+ LiveFSBuild.unique_key == unique_key,
+ LiveFSBuild.status == BuildStatus.NEEDSBUILD)
+ if pending.any() is not None:
+ raise LiveFSBuildAlreadyPending
+
+ build = getUtility(ILiveFSBuildSet).new(
+ requester, self, archive, distroarchseries, pocket,
+ unique_key=unique_key, metadata_override=metadata_override)
+ build.queueBuild()
+ return build
+
+ def _getBuilds(self, filter_term, order_by):
+ """The actual query to get the builds."""
+ query_args = [
+ LiveFSBuild.livefs == self,
+ LiveFSBuild.archive_id == Archive.id,
+ Archive._enabled == True,
+ ]
+ if filter_term is not None:
+ query_args.append(filter_term)
+ result = Store.of(self).find(LiveFSBuild, *query_args)
+ result.order_by(order_by)
+ return result
+
+ @property
+ def builds(self):
+ """See `ILiveFS`."""
+ order_by = (
+ Desc(Greatest(
+ LiveFSBuild.date_started,
+ LiveFSBuild.date_finished)),
+ Desc(LiveFSBuild.date_created),
+ Desc(LiveFSBuild.id))
+ return self._getBuilds(None, order_by)
+
+
+class LiveFSSet:
+ """See `ILiveFSSet`."""
+
+ implements(ILiveFSSet)
+
+ def new(self, registrant, owner, distroseries, name, metadata,
+ date_created=DEFAULT):
+ """See `ILiveFSSet`."""
+ store = IMasterStore(LiveFS)
+ livefs = LiveFS(
+ registrant, owner, distroseries, name, metadata, date_created)
+ store.add(livefs)
+ return livefs
+
+ def exists(self, owner, distroseries, name):
+ """See `ILiveFSSet`."""
+ livefs = self.get(owner, distroseries, name)
+ if livefs:
+ return True
+ else:
+ return False
+
+ def get(self, owner, distroseries, name):
+ """See `ILiveFSSet`."""
+ store = IMasterStore(LiveFS)
+ return store.find(
+ LiveFS,
+ LiveFS.owner == owner,
+ LiveFS.distroseries == distroseries,
+ LiveFS.name == name).one()
+
+ def _findOrRaise(self, error, name, finder, *args):
+ if name is None:
+ return None
+ args = list(args)
+ args.append(name)
+ result = finder(*args)
+ if result is None:
+ raise error(name)
+ return result
+
+ def traverse(self, segments):
+ """See `ILiveFSSet`."""
+ traversed_segments = []
+
+ def get_next_segment():
+ try:
+ result = segments.next()
+ except StopIteration:
+ raise InvalidLiveFSNamespace("/".join(traversed_segments))
+ if result is None:
+ raise AssertionError("None segment passed to traverse()")
+ traversed_segments.append(result)
+ return result
+
+ person_name = get_next_segment()
+ person = self._findOrRaise(
+ NoSuchPerson, person_name, getUtility(IPersonSet).getByName)
+ distribution_name = get_next_segment()
+ distribution = self._findOrRaise(
+ NoSuchDistribution, distribution_name,
+ getUtility(IDistributionSet).getByName)
+ distroseries_name = get_next_segment()
+ distroseries = self._findOrRaise(
+ NoSuchDistroSeries, distroseries_name,
+ getUtility(IDistroSeriesSet).queryByName, distribution)
+ livefs_name = get_next_segment()
+ return self._findOrRaise(
+ NoSuchLiveFS, livefs_name, self.get, person, distroseries)
=== added file 'lib/lp/soyuz/model/livefsbuild.py'
--- lib/lp/soyuz/model/livefsbuild.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/model/livefsbuild.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,398 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+__metaclass__ = type
+__all__ = [
+ 'LiveFSBuild',
+ 'LiveFSFile',
+ ]
+
+from datetime import timedelta
+
+import pytz
+from storm.locals import (
+ Bool,
+ DateTime,
+ Desc,
+ Int,
+ JSON,
+ Reference,
+ Store,
+ Storm,
+ Unicode,
+ )
+from storm.store import EmptyResultSet
+from zope.component import getUtility
+from zope.interface import implements
+
+from lp.app.errors import NotFoundError
+from lp.buildmaster.enums import (
+ BuildFarmJobType,
+ BuildStatus,
+ )
+from lp.buildmaster.interfaces.buildfarmjob import IBuildFarmJobSource
+from lp.buildmaster.model.buildfarmjob import SpecificBuildFarmJobSourceMixin
+from lp.buildmaster.model.packagebuild import PackageBuildMixin
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.registry.model.person import Person
+from lp.services.config import config
+from lp.services.database.bulk import load_related
+from lp.services.database.constants import DEFAULT
+from lp.services.database.decoratedresultset import DecoratedResultSet
+from lp.services.database.enumcol import DBEnum
+from lp.services.database.interfaces import (
+ IMasterStore,
+ IStore,
+ )
+from lp.services.features import getFeatureFlag
+from lp.services.librarian.browser import ProxiedLibraryFileAlias
+from lp.services.librarian.model import (
+ LibraryFileAlias,
+ LibraryFileContent,
+ )
+from lp.soyuz.interfaces.component import IComponentSet
+from lp.soyuz.interfaces.livefs import (
+ LIVEFS_FEATURE_FLAG,
+ LiveFSFeatureDisabled,
+ )
+from lp.soyuz.interfaces.livefsbuild import (
+ ILiveFSBuild,
+ ILiveFSBuildSet,
+ ILiveFSFile,
+ )
+from lp.soyuz.mail.livefsbuild import LiveFSBuildMailer
+from lp.soyuz.model.archive import Archive
+
+
+class LiveFSFile(Storm):
+ """See `ILiveFS`."""
+
+ implements(ILiveFSFile)
+
+ __storm_table__ = 'LiveFSFile'
+
+ id = Int(name='id', primary=True)
+
+ livefsbuild_id = Int(name='livefsbuild', allow_none=False)
+ livefsbuild = Reference(livefsbuild_id, 'LiveFSBuild.id')
+
+ libraryfile_id = Int(name='libraryfile', allow_none=False)
+ libraryfile = Reference(libraryfile_id, 'LibraryFileAlias.id')
+
+ def __init__(self, livefsbuild, libraryfile):
+ """Construct a `LiveFSFile`."""
+ super(LiveFSFile, self).__init__()
+ self.livefsbuild = livefsbuild
+ self.libraryfile = libraryfile
+
+
+class LiveFSBuild(PackageBuildMixin, Storm):
+ """See `ILiveFSBuild`."""
+
+ implements(ILiveFSBuild)
+
+ __storm_table__ = 'LiveFSBuild'
+
+ job_type = BuildFarmJobType.LIVEFSBUILD
+
+ id = Int(name='id', primary=True)
+
+ build_farm_job_id = Int(name='build_farm_job', allow_none=False)
+ build_farm_job = Reference(build_farm_job_id, 'BuildFarmJob.id')
+
+ requester_id = Int(name='requester', allow_none=False)
+ requester = Reference(requester_id, 'Person.id')
+
+ livefs_id = Int(name='livefs', allow_none=False)
+ livefs = Reference(livefs_id, 'LiveFS.id')
+
+ archive_id = Int(name='archive', allow_none=False)
+ archive = Reference(archive_id, 'Archive.id')
+
+ distroarchseries_id = Int(name='distroarchseries', allow_none=False)
+ distroarchseries = Reference(distroarchseries_id, 'DistroArchSeries.id')
+
+ pocket = DBEnum(enum=PackagePublishingPocket, allow_none=False)
+
+ processor_id = Int(name='processor', allow_none=False)
+ processor = Reference(processor_id, 'Processor.id')
+ virtualized = Bool(name='virtualized')
+
+ unique_key = Unicode(name='unique_key')
+
+ metadata_override = JSON('json_data_override')
+
+ date_created = DateTime(
+ name='date_created', tzinfo=pytz.UTC, allow_none=False)
+ date_started = DateTime(name='date_started', tzinfo=pytz.UTC)
+ date_finished = DateTime(name='date_finished', tzinfo=pytz.UTC)
+ date_first_dispatched = DateTime(
+ name='date_first_dispatched', tzinfo=pytz.UTC)
+
+ builder_id = Int(name='builder')
+ builder = Reference(builder_id, 'Builder.id')
+
+ status = DBEnum(name='status', enum=BuildStatus, allow_none=False)
+
+ log_id = Int(name='log')
+ log = Reference(log_id, 'LibraryFileAlias.id')
+
+ upload_log_id = Int(name='upload_log')
+ upload_log = Reference(upload_log_id, 'LibraryFileAlias.id')
+
+ dependencies = Unicode(name='dependencies')
+
+ failure_count = Int(name='failure_count', allow_none=False)
+
+ def __init__(self, build_farm_job, requester, livefs, archive,
+ distroarchseries, pocket, processor, virtualized, unique_key,
+ metadata_override, date_created):
+ """Construct a `LiveFSBuild`."""
+ if not getFeatureFlag(LIVEFS_FEATURE_FLAG):
+ raise LiveFSFeatureDisabled
+ super(LiveFSBuild, self).__init__()
+ self.build_farm_job = build_farm_job
+ self.requester = requester
+ self.livefs = livefs
+ self.archive = archive
+ self.distroarchseries = distroarchseries
+ self.pocket = pocket
+ self.processor = processor
+ self.virtualized = virtualized
+ self.unique_key = unique_key
+ if metadata_override is None:
+ metadata_override = {}
+ self.metadata_override = metadata_override
+ self.date_created = date_created
+ self.status = BuildStatus.NEEDSBUILD
+
+ @property
+ def is_private(self):
+ """See `IBuildFarmJob`."""
+ return self.livefs.owner.private or self.archive.private
+
+ @property
+ def is_virtualized(self):
+ """See `ILiveFSBuild`."""
+ return self.archive.require_virtualized
+
+ @property
+ def title(self):
+ das = self.distroarchseries
+ name = self.livefs.name
+ if self.unique_key is not None:
+ name += " (%s)" % self.unique_key
+ return "%s build of %s in %s %s %s" % (
+ das.architecturetag, name, das.distroseries.distribution.name,
+ das.distroseries.name, self.pocket.name)
+
+ @property
+ def distribution(self):
+ """See `IPackageBuild`."""
+ return self.distroarchseries.distroseries.distribution
+
+ @property
+ def distro_series(self):
+ """See `IPackageBuild`."""
+ return self.distroarchseries.distroseries
+
+ @property
+ def current_component(self):
+ component = self.archive.default_component
+ if component is not None:
+ return component
+ else:
+ # XXX cjwatson 2014-04-22: Hardcode to universe for the time being.
+ return getUtility(IComponentSet)["universe"]
+
+ @property
+ def version(self):
+ """See `ILiveFSBuild`."""
+ return self.date_created.strftime("%Y%m%d-%H%M%S")
+
+ @property
+ def score(self):
+ """See `ILiveFSBuild`."""
+ if self.buildqueue_record is None:
+ return None
+ else:
+ return self.buildqueue_record.lastscore
+
+ @property
+ def can_be_retried(self):
+ """See `ILiveFSBuild`."""
+ # We provide this property for API convenience, but live filesystem
+ # builds cannot be retried. Request another build using
+ # LiveFS.requestBuild instead.
+ return False
+
+ @property
+ def can_be_rescored(self):
+ """See `ILiveFSBuild`."""
+ return self.status is BuildStatus.NEEDSBUILD
+
+ @property
+ def can_be_cancelled(self):
+ """See `ILiveFSBuild`."""
+ if not self.buildqueue_record:
+ return False
+
+ cancellable_statuses = [
+ BuildStatus.BUILDING,
+ BuildStatus.NEEDSBUILD,
+ ]
+ return self.status in cancellable_statuses
+
+ def rescore(self, score):
+ """See `ILiveFSBuild`."""
+ assert self.can_be_rescored, "Build %s cannot be rescored" % self.id
+ self.buildqueue_record.manualScore(score)
+
+ def cancel(self):
+ """See `ILiveFSBuild`."""
+ if not self.can_be_cancelled:
+ return
+ # BuildQueue.cancel() will decide whether to go straight to
+ # CANCELLED, or go through CANCELLING to let buildd-manager clean up
+ # the slave.
+ self.buildqueue_record.cancel()
+
+ def calculateScore(self):
+ return 2505 + self.archive.relative_build_score
+
+ def getMedianBuildDuration(self):
+ """Return the median duration of builds of this live filesystem."""
+ store = IStore(self)
+ result = store.find(
+ (LiveFSBuild.date_started, LiveFSBuild.date_finished),
+ LiveFSBuild.livefs == self.livefs_id,
+ LiveFSBuild.distroarchseries == self.distroarchseries_id,
+ LiveFSBuild.date_finished != None)
+ result.order_by(Desc(LiveFSBuild.date_finished))
+ durations = [row[1] - row[0] for row in result[:9]]
+ if len(durations) == 0:
+ return None
+ durations.sort()
+ return durations[len(durations) // 2]
+
+ def estimateDuration(self):
+ """See `IBuildFarmJob`."""
+ median = self.getMedianBuildDuration()
+ if median is not None:
+ return median
+ return timedelta(minutes=30)
+
+ def getFiles(self):
+ """See `ILiveFSBuild`."""
+ store = Store.of(self)
+ result = store.find(
+ (LiveFSFile, LibraryFileAlias, LibraryFileContent),
+ LiveFSFile.livefsbuild == self.id,
+ LibraryFileAlias.id == LiveFSFile.libraryfile_id,
+ LibraryFileContent.id == LibraryFileAlias.contentID)
+ return result.order_by(
+ [LibraryFileAlias.filename, LiveFSFile.id]).config(distinct=True)
+
+ def getFileByName(self, filename):
+ """See `ILiveFSBuild`."""
+ if filename.endswith(".txt.gz"):
+ file_object = self.log
+ elif filename.endswith("_log.txt"):
+ file_object = self.upload_log
+ else:
+ file_object = Store.of(self).find(
+ LibraryFileAlias,
+ LiveFSFile.livefsbuild == self.id,
+ LibraryFileAlias.id == LiveFSFile.libraryfile_id,
+ LibraryFileAlias.filename == filename).one()
+
+ if file_object is not None and file_object.filename == filename:
+ return file_object
+
+ raise NotFoundError(filename)
+
+ def addFile(self, lfa):
+ """See `ILiveFSBuild`."""
+ return LiveFSFile(livefsbuild=self, libraryfile=lfa)
+
+ def verifySuccessfulUpload(self):
+ """See `IPackageBuild`."""
+ return not self.getFiles().is_empty()
+
+ def notify(self, extra_info=None):
+ """See `IPackageBuild`."""
+ if not config.builddmaster.send_build_notification:
+ return
+ if self.status == BuildStatus.FULLYBUILT:
+ return
+ mailer = LiveFSBuildMailer.forStatus(self)
+ mailer.sendAll()
+
+ def getUploader(self, changes):
+ """See `IPackageBuild`."""
+ return self.requester
+
+ def lfaUrl(self, lfa):
+ """Return the URL for a LibraryFileAlias in this context."""
+ if lfa is None:
+ return None
+ return ProxiedLibraryFileAlias(lfa, self).http_url
+
+ @property
+ def log_url(self):
+ """See `IBuildFarmJob`."""
+ return self.lfaUrl(self.log)
+
+ @property
+ def upload_log_url(self):
+ """See `IPackageBuild`."""
+ return self.lfaUrl(self.upload_log)
+
+ def getFileUrls(self):
+ return [self.lfaUrl(lfa) for _, lfa, _ in self.getFiles()]
+
+
+class LiveFSBuildSet(SpecificBuildFarmJobSourceMixin):
+ implements(ILiveFSBuildSet)
+
+ def new(self, requester, livefs, archive, distroarchseries, pocket,
+ unique_key=None, metadata_override=None, date_created=DEFAULT):
+ """See `ILiveFSBuildSet`."""
+ store = IMasterStore(LiveFSBuild)
+ build_farm_job = getUtility(IBuildFarmJobSource).new(
+ LiveFSBuild.job_type, BuildStatus.NEEDSBUILD, date_created, None,
+ archive)
+ livefsbuild = LiveFSBuild(
+ build_farm_job, requester, livefs, archive, distroarchseries,
+ pocket, distroarchseries.processor, archive.require_virtualized,
+ unique_key, metadata_override, date_created)
+ store.add(livefsbuild)
+ return livefsbuild
+
+ def getByID(self, build_id):
+ """See `ISpecificBuildFarmJobSource`."""
+ store = IMasterStore(LiveFSBuild)
+ return store.find(LiveFSBuild, LiveFSBuild.id == build_id).one()
+
+ def getByBuildFarmJob(self, build_farm_job):
+ """See `ISpecificBuildFarmJobSource`."""
+ return Store.of(build_farm_job).find(
+ LiveFSBuild, build_farm_job_id=build_farm_job.id).one()
+
+ def preloadBuildsData(self, builds):
+ # Circular import.
+ from lp.soyuz.model.livefs import LiveFS
+ load_related(Person, builds, ["requester_id"])
+ load_related(LibraryFileAlias, builds, ["log_id"])
+ archives = load_related(Archive, builds, ["archive_id"])
+ load_related(Person, archives, ["ownerID"])
+ load_related(LiveFS, builds, ["livefs_id"])
+
+ def getByBuildFarmJobs(self, build_farm_jobs):
+ """See `ISpecificBuildFarmJobSource`."""
+ if len(build_farm_jobs) == 0:
+ return EmptyResultSet()
+ rows = Store.of(build_farm_jobs[0]).find(
+ LiveFSBuild, LiveFSBuild.build_farm_job_id.is_in(
+ bfj.id for bfj in build_farm_jobs))
+ return DecoratedResultSet(rows, pre_iter_hook=self.preloadBuildsData)
=== added file 'lib/lp/soyuz/model/livefsbuildbehaviour.py'
--- lib/lp/soyuz/model/livefsbuildbehaviour.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/model/livefsbuildbehaviour.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,133 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""An `IBuildFarmJobBehaviour` for `LiveFSBuild`.
+
+Dispatches live filesystem build jobs to build-farm slaves.
+"""
+
+__metaclass__ = type
+__all__ = [
+ 'LiveFSBuildBehaviour',
+ ]
+
+from twisted.internet import defer
+from zope.component import adapts
+from zope.interface import implements
+
+from lp.buildmaster.interfaces.builder import CannotBuild
+from lp.buildmaster.interfaces.buildfarmjobbehaviour import (
+ IBuildFarmJobBehaviour,
+ )
+from lp.buildmaster.model.buildfarmjobbehaviour import (
+ BuildFarmJobBehaviourBase,
+ )
+from lp.soyuz.adapters.archivedependencies import get_sources_list_for_building
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuild
+
+
+class LiveFSBuildBehaviour(BuildFarmJobBehaviourBase):
+ """Dispatches `LiveFSBuild` jobs to slaves."""
+
+ adapts(ILiveFSBuild)
+ implements(IBuildFarmJobBehaviour)
+
+ # Identify the type of job to the slave.
+ build_type = 'livefs'
+
+ @property
+ def displayname(self):
+ ret = self.build.title
+ if self._builder is not None:
+ ret += " (on %s)" % self._builder.url
+ return ret
+
+ def logStartBuild(self, logger):
+ """See `IBuildFarmJobBehaviour`."""
+ logger.info("startBuild(%s)", self.displayname)
+
+ def getLogFileName(self):
+ das = self.build.distroarchseries
+ archname = das.architecturetag
+ if self.build.unique_key:
+ archname += '_%s' % self.build.unique_key
+
+ # Examples:
+ # buildlog_ubuntu_trusty_i386_ubuntu-desktop_FULLYBUILT.txt
+ return 'buildlog_%s_%s_%s_%s_%s.txt' % (
+ das.distroseries.distribution.name, das.distroseries.name,
+ archname, self.build.livefs.name, self.build.status.name)
+
+ def verifyBuildRequest(self, logger):
+ """Assert some pre-build checks.
+
+ The build request is checked:
+ * Virtualized builds can't build on a non-virtual builder
+ * Ensure that we have a chroot
+ """
+ build = self.build
+ if build.is_virtualized and not self._builder.virtualized:
+ raise AssertionError(
+ "Attempt to build virtual item on a non-virtual builder.")
+
+ chroot = build.distroarchseries.getChroot()
+ if chroot is None:
+ raise CannotBuild(
+ "Missing chroot for %s" % build.distroarchseries.displayname)
+
+ def _extraBuildArgs(self):
+ """
+ Return the extra arguments required by the slave for the given build.
+ """
+ build = self.build
+ args = dict(build.livefs.metadata)
+ args.update(build.metadata_override)
+ args["suite"] = build.distroarchseries.distroseries.getSuite(
+ build.pocket)
+ args["arch_tag"] = build.distroarchseries.architecturetag
+ args["datestamp"] = build.version
+ args["archives"] = get_sources_list_for_building(
+ build, build.distroarchseries, None)
+ args["archive_private"] = build.archive.private
+ return args
+
+ @defer.inlineCallbacks
+ def dispatchBuildToSlave(self, build_queue_id, logger):
+ """See `IBuildFarmJobBehaviour`."""
+
+ # Start the build on the slave builder. First we send the chroot.
+ distroarchseries = self.build.distroarchseries
+ chroot = distroarchseries.getChroot()
+ if chroot is None:
+ raise CannotBuild(
+ "Unable to find a chroot for %s" %
+ distroarchseries.displayname)
+ logger.info(
+ "Sending chroot file for live filesystem build to %s" %
+ self._builder.name)
+ yield self._slave.cacheFile(logger, chroot)
+
+ # Generate a string which can be used to cross-check when obtaining
+ # results so we know we are referring to the right database object
+ # in subsequent runs.
+ buildid = "%s-%s" % (self.build.id, build_queue_id)
+ logger.info("Initiating build %s on %s" % (buildid, self._builder.url))
+
+ cookie = self.getBuildCookie()
+ args = self._extraBuildArgs()
+ status, info = yield self._slave.build(
+ cookie, "livefs", chroot.content.sha1, {}, args)
+
+ message = """%s (%s):
+ ***** RESULT *****
+ %s
+ %s: %s
+ ******************
+ """ % (
+ self._builder.name,
+ self._builder.url,
+ args,
+ status,
+ info,
+ )
+ logger.info(message)
=== added file 'lib/lp/soyuz/tests/test_livefs.py'
--- lib/lp/soyuz/tests/test_livefs.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/tests/test_livefs.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,377 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test live filesystems."""
+
+__metaclass__ = type
+
+from datetime import timedelta
+
+from storm.locals import Store
+from testtools.matchers import Equals
+import transaction
+from zope.component import getUtility
+from zope.security.proxy import removeSecurityProxy
+
+from lp.buildmaster.enums import (
+ BuildQueueStatus,
+ BuildStatus,
+ )
+from lp.buildmaster.interfaces.buildqueue import IBuildQueue
+from lp.buildmaster.model.buildqueue import BuildQueue
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.services.features.testing import FeatureFixture
+from lp.services.webapp.interfaces import OAuthPermission
+from lp.soyuz.interfaces.livefs import (
+ ILiveFS,
+ ILiveFSSet,
+ ILiveFSView,
+ LIVEFS_FEATURE_FLAG,
+ LiveFSBuildAlreadyPending,
+ LiveFSFeatureDisabled,
+ )
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuild
+from lp.testing import (
+ ANONYMOUS,
+ api_url,
+ login,
+ logout,
+ person_logged_in,
+ StormStatementRecorder,
+ TestCaseWithFactory,
+ )
+from lp.testing.layers import (
+ DatabaseFunctionalLayer,
+ LaunchpadZopelessLayer,
+ )
+from lp.testing.matchers import (
+ DoesNotSnapshot,
+ HasQueryCount,
+ )
+from lp.testing.pages import webservice_for_person
+
+
+class TestLiveFSFeatureFlag(TestCaseWithFactory):
+
+ layer = LaunchpadZopelessLayer
+
+ def test_feature_flag_disabled(self):
+ # Without a feature flag, we will not create new LiveFSes.
+ self.assertRaises(
+ LiveFSFeatureDisabled, getUtility(ILiveFSSet).new,
+ None, None, None, None, None)
+
+
+class TestLiveFS(TestCaseWithFactory):
+
+ layer = DatabaseFunctionalLayer
+
+ def setUp(self):
+ super(TestLiveFS, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+
+ def test_implements_interfaces(self):
+ # LiveFS implements ILiveFS.
+ livefs = self.factory.makeLiveFS()
+ self.assertProvides(livefs, ILiveFS)
+
+ def test_class_implements_interfaces(self):
+ # The LiveFS class implements ILiveFSSet.
+ self.assertProvides(getUtility(ILiveFSSet), ILiveFSSet)
+
+ def test_avoids_problematic_snapshot(self):
+ self.assertThat(
+ self.factory.makeLiveFS(),
+ DoesNotSnapshot(["builds"], ILiveFSView))
+
+ def makeLiveFSComponents(self, metadata={}):
+ """Return a dict of values that can be used to make a LiveFS.
+
+ Suggested use: provide as kwargs to ILiveFSSet.new.
+
+ :param metadata: A dict to set as LiveFS.metadata.
+ """
+ registrant = self.factory.makePerson()
+ return dict(
+ registrant=registrant,
+ owner=self.factory.makeTeam(owner=registrant),
+ distroseries=self.factory.makeDistroSeries(),
+ name=self.factory.getUniqueString(u"livefs-name"),
+ metadata=metadata)
+
+ def test_creation(self):
+ # The metadata entries supplied when a LiveFS is created are present
+ # on the new object.
+ components = self.makeLiveFSComponents(metadata={"project": "foo"})
+ livefs = getUtility(ILiveFSSet).new(**components)
+ transaction.commit()
+ self.assertEqual(components["registrant"], livefs.registrant)
+ self.assertEqual(components["owner"], livefs.owner)
+ self.assertEqual(components["distroseries"], livefs.distroseries)
+ self.assertEqual(components["name"], livefs.name)
+ self.assertEqual(components["metadata"], livefs.metadata)
+
+ def test_exists(self):
+ # ILiveFSSet.exists checks for matching LiveFSes.
+ livefs = self.factory.makeLiveFS()
+ self.assertTrue(
+ getUtility(ILiveFSSet).exists(
+ livefs.owner, livefs.distroseries, livefs.name))
+ self.assertFalse(
+ getUtility(ILiveFSSet).exists(
+ self.factory.makePerson(), livefs.distroseries, livefs.name))
+ self.assertFalse(
+ getUtility(ILiveFSSet).exists(
+ livefs.owner, self.factory.makeDistroSeries(), livefs.name))
+ self.assertFalse(
+ getUtility(ILiveFSSet).exists(
+ livefs.owner, livefs.distroseries, u"different"))
+
+ def test_requestBuild(self):
+ # requestBuild creates a new LiveFSBuild.
+ livefs = self.factory.makeLiveFS()
+ requester = self.factory.makePerson()
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=livefs.distroseries)
+ build = livefs.requestBuild(
+ requester, livefs.distroseries.main_archive, distroarchseries,
+ PackagePublishingPocket.RELEASE)
+ self.assertTrue(ILiveFSBuild.providedBy(build))
+ self.assertEqual(requester, build.requester)
+ self.assertEqual(livefs.distroseries.main_archive, build.archive)
+ self.assertEqual(distroarchseries, build.distroarchseries)
+ self.assertEqual(PackagePublishingPocket.RELEASE, build.pocket)
+ self.assertIsNone(build.unique_key)
+ self.assertEqual({}, build.metadata_override)
+ self.assertEqual(BuildStatus.NEEDSBUILD, build.status)
+ store = Store.of(build)
+ store.flush()
+ build_queue = store.find(
+ BuildQueue,
+ BuildQueue._build_farm_job_id ==
+ removeSecurityProxy(build).build_farm_job_id).one()
+ self.assertProvides(build_queue, IBuildQueue)
+ self.assertEqual(
+ livefs.distroseries.main_archive.require_virtualized,
+ build_queue.virtualized)
+ self.assertEqual(BuildQueueStatus.WAITING, build_queue.status)
+
+ def test_requestBuild_score(self):
+ # Build requests have a relatively low queue score (2505).
+ livefs = self.factory.makeLiveFS()
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=livefs.distroseries)
+ build = livefs.requestBuild(
+ livefs.owner, livefs.distroseries.main_archive, distroarchseries,
+ PackagePublishingPocket.RELEASE)
+ queue_record = build.buildqueue_record
+ queue_record.score()
+ self.assertEqual(2505, queue_record.lastscore)
+
+ def test_requestBuild_relative_build_score(self):
+ # Offsets for archives are respected.
+ livefs = self.factory.makeLiveFS()
+ archive = self.factory.makeArchive(owner=livefs.owner)
+ removeSecurityProxy(archive).relative_build_score = 100
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=livefs.distroseries)
+ build = livefs.requestBuild(
+ livefs.owner, archive, distroarchseries,
+ PackagePublishingPocket.RELEASE)
+ queue_record = build.buildqueue_record
+ queue_record.score()
+ self.assertEqual(2605, queue_record.lastscore)
+
+ def test_requestBuild_rejects_repeats(self):
+ # requestBuild refuses if there is already a pending build.
+ livefs = self.factory.makeLiveFS()
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=livefs.distroseries)
+ old_build = livefs.requestBuild(
+ livefs.owner, livefs.distroseries.main_archive, distroarchseries,
+ PackagePublishingPocket.RELEASE)
+ self.assertRaises(
+ LiveFSBuildAlreadyPending, livefs.requestBuild,
+ livefs.owner, livefs.distroseries.main_archive, distroarchseries,
+ PackagePublishingPocket.RELEASE)
+ # We can build for a different archive.
+ livefs.requestBuild(
+ livefs.owner, self.factory.makeArchive(owner=livefs.owner),
+ distroarchseries, PackagePublishingPocket.RELEASE)
+ # We can build for a different distroarchseries.
+ livefs.requestBuild(
+ livefs.owner, livefs.distroseries.main_archive,
+ self.factory.makeDistroArchSeries(
+ distroseries=livefs.distroseries),
+ PackagePublishingPocket.RELEASE)
+ # Changing the status of the old build allows a new build.
+ old_build.updateStatus(BuildStatus.FULLYBUILT)
+ livefs.requestBuild(
+ livefs.owner, livefs.distroseries.main_archive, distroarchseries,
+ PackagePublishingPocket.RELEASE)
+
+ def test_getBuilds(self):
+ # Test the various getBuilds methods.
+ livefs = self.factory.makeLiveFS()
+ builds = [
+ self.factory.makeLiveFSBuild(livefs=livefs) for x in range(3)]
+ # We want the latest builds first.
+ builds.reverse()
+
+ self.assertEqual(builds, list(livefs.builds))
+
+ # Change the status of one of the builds and retest.
+ builds[0].updateStatus(BuildStatus.FULLYBUILT)
+ self.assertEqual(builds, list(livefs.builds))
+
+
+class TestLiveFSWebservice(TestCaseWithFactory):
+
+ layer = DatabaseFunctionalLayer
+
+ def setUp(self):
+ super(TestLiveFSWebservice, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+ self.person = self.factory.makePerson()
+ self.webservice = webservice_for_person(
+ self.person, permission=OAuthPermission.WRITE_PUBLIC)
+ self.webservice.default_api_version = "devel"
+ login(ANONYMOUS)
+
+ def getURL(self, obj):
+ return self.webservice.getAbsoluteUrl(api_url(obj))
+
+ def makeLiveFS(self, registrant=None, owner=None, distroseries=None,
+ metadata=None):
+ if registrant is None:
+ registrant = self.person
+ if owner is None:
+ owner = registrant
+ if metadata is None:
+ metadata = {"project": "flavour"}
+ if distroseries is None:
+ distroseries = self.factory.makeDistroSeries(registrant=registrant)
+ transaction.commit()
+ distroseries_url = api_url(distroseries)
+ registrant_url = api_url(registrant)
+ owner_url = api_url(owner)
+ logout()
+ response = self.webservice.named_post(
+ registrant_url, "createLiveFS", owner=owner_url,
+ distroseries=distroseries_url, name="flavour-desktop",
+ metadata=metadata)
+ self.assertEqual(201, response.status)
+ livefs = self.webservice.get(response.getHeader("Location")).jsonBody()
+ return livefs, distroseries_url
+
+ def getCollectionLinks(self, entry, member):
+ """Return a list of self_link attributes of entries in a collection."""
+ collection = self.webservice.get(
+ entry["%s_collection_link" % member]).jsonBody()
+ return [entry["self_link"] for entry in collection["entries"]]
+
+ def test_createLiveFS(self):
+ # Ensure LiveFS creation works.
+ team = self.factory.makeTeam(owner=self.person)
+ livefs, distroseries_url = self.makeLiveFS(owner=team)
+ with person_logged_in(self.person):
+ self.assertEqual(
+ self.getURL(self.person), livefs["registrant_link"])
+ self.assertEqual(self.getURL(team), livefs["owner_link"])
+ self.assertEqual("flavour-desktop", livefs["name"])
+ self.assertEqual(
+ self.webservice.getAbsoluteUrl(distroseries_url),
+ livefs["distroseries_link"])
+ self.assertEqual({"project": "flavour"}, livefs["metadata"])
+
+ def test_requestBuild(self):
+ # Build requests can be performed and end up in livefs.builds.
+ distroseries = self.factory.makeDistroSeries(registrant=self.person)
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=distroseries, owner=self.person)
+ distroarchseries_url = api_url(distroarchseries)
+ archive_url = api_url(distroseries.main_archive)
+ livefs, distroseries_url = self.makeLiveFS(distroseries=distroseries)
+ response = self.webservice.named_post(
+ livefs["self_link"], "requestBuild", archive=archive_url,
+ distroarchseries=distroarchseries_url, pocket="Release")
+ self.assertEqual(201, response.status)
+ build = self.webservice.get(response.getHeader("Location")).jsonBody()
+ self.assertEqual(
+ [build["self_link"]], self.getCollectionLinks(livefs, "builds"))
+
+ def test_requestBuild_rejects_repeats(self):
+ # Build requests are rejected if already pending.
+ distroseries = self.factory.makeDistroSeries(registrant=self.person)
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=distroseries, owner=self.person)
+ distroarchseries_url = api_url(distroarchseries)
+ archive_url = api_url(distroseries.main_archive)
+ livefs, ws_distroseries = self.makeLiveFS(distroseries=distroseries)
+ response = self.webservice.named_post(
+ livefs["self_link"], "requestBuild", archive=archive_url,
+ distroarchseries=distroarchseries_url, pocket="Release")
+ self.assertEqual(201, response.status)
+ response = self.webservice.named_post(
+ livefs["self_link"], "requestBuild", archive=archive_url,
+ distroarchseries=distroarchseries_url, pocket="Release")
+ self.assertEqual(400, response.status)
+ self.assertEqual(
+ "An identical build of this live filesystem image is already "
+ "pending.", response.body)
+
+ def test_getBuilds(self):
+ # The builds property is as expected.
+ distroseries = self.factory.makeDistroSeries(registrant=self.person)
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=distroseries, owner=self.person)
+ distroarchseries_url = api_url(distroarchseries)
+ archives = [
+ self.factory.makeArchive(
+ distribution=distroseries.distribution, owner=self.person)
+ for x in range(4)]
+ archive_urls = [api_url(archive) for archive in archives]
+ livefs, distroseries_url = self.makeLiveFS(distroseries=distroseries)
+ builds = []
+ for archive_url in archive_urls:
+ response = self.webservice.named_post(
+ livefs["self_link"], "requestBuild", archive=archive_url,
+ distroarchseries=distroarchseries_url, pocket="Proposed")
+ self.assertEqual(201, response.status)
+ build = self.webservice.get(
+ response.getHeader("Location")).jsonBody()
+ builds.insert(0, build["self_link"])
+ self.assertEqual(builds, self.getCollectionLinks(livefs, "builds"))
+ livefs = self.webservice.get(livefs["self_link"]).jsonBody()
+
+ with person_logged_in(self.person):
+ db_livefs = getUtility(ILiveFSSet).get(
+ self.person, distroseries, livefs["name"])
+ db_builds = list(db_livefs.builds)
+ db_builds[0].updateStatus(
+ BuildStatus.BUILDING, date_started=db_livefs.date_created)
+ db_builds[0].updateStatus(
+ BuildStatus.FULLYBUILT,
+ date_finished=db_livefs.date_created + timedelta(minutes=10))
+ livefs = self.webservice.get(livefs["self_link"]).jsonBody()
+
+ with person_logged_in(self.person):
+ db_builds[1].updateStatus(
+ BuildStatus.BUILDING, date_started=db_livefs.date_created)
+ db_builds[1].updateStatus(
+ BuildStatus.FULLYBUILT,
+ date_finished=db_livefs.date_created + timedelta(minutes=20))
+ livefs = self.webservice.get(livefs["self_link"]).jsonBody()
+
+ def test_query_count(self):
+ # LiveFS has a reasonable query count.
+ livefs = self.factory.makeLiveFS(
+ registrant=self.person, owner=self.person)
+ url = api_url(livefs)
+ logout()
+ store = Store.of(livefs)
+ store.flush()
+ store.invalidate()
+ with StormStatementRecorder() as recorder:
+ self.webservice.get(url)
+ self.assertThat(recorder, HasQueryCount(Equals(21)))
=== added file 'lib/lp/soyuz/tests/test_livefsbuild.py'
--- lib/lp/soyuz/tests/test_livefsbuild.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/tests/test_livefsbuild.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,481 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test live filesystem build features."""
+
+__metaclass__ = type
+
+from datetime import (
+ datetime,
+ timedelta,
+ )
+from urllib2 import (
+ HTTPError,
+ urlopen,
+ )
+
+import pytz
+from zope.component import getUtility
+from zope.security.proxy import removeSecurityProxy
+from zope.testbrowser.browser import Browser
+from zope.testbrowser.testing import PublisherMechanizeBrowser
+
+from lp.app.errors import NotFoundError
+from lp.app.interfaces.launchpad import ILaunchpadCelebrities
+from lp.buildmaster.enums import BuildStatus
+from lp.buildmaster.interfaces.buildqueue import IBuildQueue
+from lp.buildmaster.interfaces.packagebuild import IPackageBuild
+from lp.registry.enums import PersonVisibility
+from lp.services.config import config
+from lp.services.features.testing import FeatureFixture
+from lp.services.librarian.browser import ProxiedLibraryFileAlias
+from lp.services.webapp.interfaces import OAuthPermission
+from lp.soyuz.enums import ArchivePurpose
+from lp.soyuz.interfaces.livefs import (
+ LIVEFS_FEATURE_FLAG,
+ LiveFSFeatureDisabled,
+ )
+from lp.soyuz.interfaces.livefsbuild import (
+ ILiveFSBuild,
+ ILiveFSBuildSet,
+ )
+from lp.soyuz.interfaces.processor import IProcessorSet
+from lp.testing import (
+ ANONYMOUS,
+ api_url,
+ login,
+ logout,
+ person_logged_in,
+ TestCaseWithFactory,
+ )
+from lp.testing.layers import (
+ LaunchpadFunctionalLayer,
+ LaunchpadZopelessLayer,
+ )
+from lp.testing.mail_helpers import pop_notifications
+from lp.testing.pages import webservice_for_person
+
+
+class TestLiveFSBuildFeatureFlag(TestCaseWithFactory):
+
+ layer = LaunchpadZopelessLayer
+
+ def test_feature_flag_disabled(self):
+ # Without a feature flag, we will not create new LiveFSBuilds.
+ self.assertRaises(
+ LiveFSFeatureDisabled, getUtility(ILiveFSBuildSet).new,
+ None, None, self.factory.makeArchive(),
+ self.factory.makeDistroArchSeries(), None, None, None)
+
+
+expected_body = """\
+ * Live Filesystem: livefs-1
+ * Version: 20140425-103800
+ * Archive: distro primary archive
+ * Distroseries: distro unstable
+ * Architecture: i386
+ * Pocket: RELEASE
+ * State: Failed to build
+ * Duration: 10 minutes
+ * Build Log: %s
+ * Upload Log: %s
+ * Builder: http://launchpad.dev/builders/bob
+"""
+
+
+class TestLiveFSBuild(TestCaseWithFactory):
+
+ layer = LaunchpadZopelessLayer
+
+ def setUp(self):
+ super(TestLiveFSBuild, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+ self.build = self.factory.makeLiveFSBuild()
+
+ def test_implements_interfaces(self):
+ # LiveFSBuild implements IPackageBuild and ILiveFSBuild.
+ self.assertProvides(self.build, IPackageBuild)
+ self.assertProvides(self.build, ILiveFSBuild)
+
+ def test_queueBuild(self):
+ # LiveFSBuild can create the queue entry for itself.
+ bq = self.build.queueBuild()
+ self.assertProvides(bq, IBuildQueue)
+ self.assertEqual(
+ self.build.build_farm_job, removeSecurityProxy(bq)._build_farm_job)
+ self.assertEqual(self.build, bq.specific_build)
+ self.assertEqual(self.build.virtualized, bq.virtualized)
+ self.assertIsNotNone(bq.processor)
+ self.assertEqual(bq, self.build.buildqueue_record)
+
+ def test_current_component_primary(self):
+ # LiveFSBuilds for primary archives always build in universe for the
+ # time being.
+ self.assertEqual(ArchivePurpose.PRIMARY, self.build.archive.purpose)
+ self.assertEqual("universe", self.build.current_component.name)
+
+ def test_current_component_ppa(self):
+ # PPAs only have indices for main, so LiveFSBuilds for PPAs always
+ # build in main.
+ build = self.factory.makeLiveFSBuild(
+ archive=self.factory.makeArchive())
+ self.assertEqual("main", build.current_component.name)
+
+ def test_is_private(self):
+ # A LiveFSBuild is private iff its LiveFS and archive are.
+ self.assertFalse(self.build.is_private)
+ private_team = self.factory.makeTeam(
+ visibility=PersonVisibility.PRIVATE)
+ with person_logged_in(private_team.teamowner):
+ build = self.factory.makeLiveFSBuild(owner=private_team)
+ self.assertTrue(build.is_private)
+ private_archive = self.factory.makeArchive(private=True)
+ with person_logged_in(private_archive.owner):
+ build = self.factory.makeLiveFSBuild(archive=private_archive)
+ self.assertTrue(build.is_private)
+
+ def test_can_be_cancelled(self):
+ # For all states that can be cancelled, can_be_cancelled returns True.
+ ok_cases = [
+ BuildStatus.BUILDING,
+ BuildStatus.NEEDSBUILD,
+ ]
+ for status in BuildStatus:
+ if status in ok_cases:
+ self.assertTrue(self.build.can_be_cancelled)
+ else:
+ self.assertFalse(self.build.can_be_cancelled)
+
+ def test_cancel_not_in_progress(self):
+ # The cancel() method for a pending build leaves it in the CANCELLED
+ # state.
+ self.build.queueBuild()
+ self.build.cancel()
+ self.assertEqual(BuildStatus.CANCELLED, self.build.status)
+ self.assertIsNone(self.build.buildqueue_record)
+
+ def test_cancel_in_progress(self):
+ # The cancel() method for a building build leaves it in the
+ # CANCELLING state.
+ bq = self.build.queueBuild()
+ bq.markAsBuilding(self.factory.makeBuilder())
+ self.build.cancel()
+ self.assertEqual(BuildStatus.CANCELLING, self.build.status)
+ self.assertEqual(bq, self.build.buildqueue_record)
+
+ def test_estimateDuration(self):
+ # Without previous builds, the default time estimate is 30m.
+ self.assertEqual(1800, self.build.estimateDuration().seconds)
+
+ def test_estimateDuration_with_history(self):
+ # Previous builds of the same live filesystem are used for estimates.
+ self.factory.makeLiveFSBuild(
+ requester=self.build.requester, livefs=self.build.livefs,
+ distroarchseries=self.build.distroarchseries,
+ status=BuildStatus.FULLYBUILT, duration=timedelta(seconds=335))
+ self.assertEqual(335, self.build.estimateDuration().seconds)
+
+ def test_getFileByName_logs(self):
+ # getFileByName returns the logs when requested by name.
+ self.build.setLog(
+ self.factory.makeLibraryFileAlias(filename="buildlog.txt.gz"))
+ self.assertEqual(
+ self.build.log, self.build.getFileByName("buildlog.txt.gz"))
+ self.assertRaises(NotFoundError, self.build.getFileByName, "foo")
+ self.build.storeUploadLog("uploaded")
+ self.assertEqual(
+ self.build.upload_log,
+ self.build.getFileByName(self.build.upload_log.filename))
+
+ def test_getFileByName_uploaded_files(self):
+ # getFileByName returns uploaded files when requested by name.
+ filenames = ("ubuntu.squashfs", "ubuntu.manifest")
+ lfas = []
+ for filename in filenames:
+ lfa = self.factory.makeLibraryFileAlias(filename=filename)
+ lfas.append(lfa)
+ self.factory.makeLiveFSFile(
+ livefsbuild=self.build, libraryfile=lfa)
+ self.assertContentEqual(
+ lfas, [row[1] for row in self.build.getFiles()])
+ for filename, lfa in zip(filenames, lfas):
+ self.assertEqual(lfa, self.build.getFileByName(filename))
+ self.assertRaises(NotFoundError, self.build.getFileByName, "missing")
+
+ def test_verifySuccessfulUpload(self):
+ self.assertFalse(self.build.verifySuccessfulUpload())
+ self.factory.makeLiveFSFile(livefsbuild=self.build)
+ self.assertTrue(self.build.verifySuccessfulUpload())
+
+ def test_notify_fullybuilt(self):
+ # notify does not send mail when a LiveFSBuild completes normally.
+ person = self.factory.makePerson(name="person")
+ build = self.factory.makeLiveFSBuild(
+ requester=person, status=BuildStatus.FULLYBUILT)
+ build.notify()
+ self.assertEqual(0, len(pop_notifications()))
+
+ def test_notify_packagefail(self):
+ # notify sends mail when a LiveFSBuild fails.
+ person = self.factory.makePerson(name="person")
+ distribution = self.factory.makeDistribution(name="distro")
+ distroseries = self.factory.makeDistroSeries(
+ distribution=distribution, name="unstable")
+ processor = getUtility(IProcessorSet).getByName("386")
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=distroseries, architecturetag="i386",
+ processor=processor)
+ build = self.factory.makeLiveFSBuild(
+ name=u"livefs-1", requester=person,
+ distroarchseries=distroarchseries,
+ date_created=datetime(2014, 04, 25, 10, 38, 0, tzinfo=pytz.UTC),
+ status=BuildStatus.FAILEDTOBUILD,
+ builder=self.factory.makeBuilder(name="bob"),
+ duration=timedelta(minutes=10))
+ build.setLog(self.factory.makeLibraryFileAlias())
+ build.notify()
+ [notification] = pop_notifications()
+ self.assertEqual(
+ config.canonical.noreply_from_address, notification["From"])
+ self.assertEqual(
+ "Person <%s>" % person.preferredemail.email, notification["To"])
+ self.assertEqual(
+ "[LiveFS build #%d] i386 build of livefs-1 in distro unstable "
+ "RELEASE" % build.id, notification["Subject"])
+ self.assertEqual(
+ "Requester", notification["X-Launchpad-Message-Rationale"])
+ self.assertEqual(
+ "livefs-build-status",
+ notification["X-Launchpad-Notification-Type"])
+ self.assertEqual(
+ "FAILEDTOBUILD", notification["X-Launchpad-Build-State"])
+ body, footer = notification.get_payload(decode=True).split("\n-- \n")
+ self.assertEqual(expected_body % (build.log_url, ""), body)
+ self.assertEqual(
+ "http://launchpad.dev/distro/+archive/primary/+livefsbuild/%d\n";
+ "You are the requester of the build.\n" % build.id, footer)
+
+ def addFakeBuildLog(self, build):
+ build.setLog(self.factory.makeLibraryFileAlias("mybuildlog.txt"))
+
+ def test_log_url(self):
+ # The log URL for a live filesystem build will use the archive context.
+ self.addFakeBuildLog(self.build)
+ self.assertEqual(
+ "http://launchpad.dev/%s/+archive/primary/+livefsbuild/%d/+files/";
+ "mybuildlog.txt" % (self.build.distribution.name, self.build.id),
+ self.build.log_url)
+
+
+class TestLiveFSBuildSet(TestCaseWithFactory):
+
+ layer = LaunchpadZopelessLayer
+
+ def setUp(self):
+ super(TestLiveFSBuildSet, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+
+ def test_getByBuildFarmJob_works(self):
+ build = self.factory.makeLiveFSBuild()
+ self.assertEqual(
+ build,
+ getUtility(ILiveFSBuildSet).getByBuildFarmJob(
+ build.build_farm_job))
+
+ def test_getByBuildFarmJob_returns_None_when_missing(self):
+ bpb = self.factory.makeBinaryPackageBuild()
+ self.assertIsNone(
+ getUtility(ILiveFSBuildSet).getByBuildFarmJob(bpb.build_farm_job))
+
+ def test_getByBuildFarmJobs_works(self):
+ builds = [self.factory.makeLiveFSBuild() for i in range(10)]
+ self.assertContentEqual(
+ builds,
+ getUtility(ILiveFSBuildSet).getByBuildFarmJobs(
+ [build.build_farm_job for build in builds]))
+
+ def test_getByBuildFarmJobs_works_empty(self):
+ self.assertContentEqual(
+ [], getUtility(ILiveFSBuildSet).getByBuildFarmJobs([]))
+
+
+class NonRedirectingMechanizeBrowser(PublisherMechanizeBrowser):
+ """A `mechanize.Browser` that does not handle redirects."""
+
+ default_features = [
+ feature for feature in PublisherMechanizeBrowser.default_features
+ if feature != "_redirect"]
+
+
+class TestLiveFSBuildWebservice(TestCaseWithFactory):
+
+ layer = LaunchpadFunctionalLayer
+
+ def setUp(self):
+ super(TestLiveFSBuildWebservice, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+ self.person = self.factory.makePerson()
+ self.webservice = webservice_for_person(
+ self.person, permission=OAuthPermission.WRITE_PUBLIC)
+ self.webservice.default_api_version = "devel"
+ login(ANONYMOUS)
+
+ def getURL(self, obj):
+ return self.webservice.getAbsoluteUrl(api_url(obj))
+
+ def test_properties(self):
+ # The basic properties of a LiveFSBuild are sensible.
+ db_build = self.factory.makeLiveFSBuild(
+ requester=self.person, unique_key=u"foo",
+ metadata_override={"image_format": "plain"},
+ date_created=datetime(2014, 04, 25, 10, 38, 0, tzinfo=pytz.UTC))
+ build_url = api_url(db_build)
+ logout()
+ build = self.webservice.get(build_url).jsonBody()
+ with person_logged_in(self.person):
+ self.assertEqual(self.getURL(self.person), build["requester_link"])
+ self.assertEqual(
+ self.getURL(db_build.livefs), build["livefs_link"])
+ self.assertEqual(
+ self.getURL(db_build.archive), build["archive_link"])
+ self.assertEqual(
+ self.getURL(db_build.distroarchseries),
+ build["distroarchseries_link"])
+ self.assertEqual("Release", build["pocket"])
+ self.assertEqual("foo", build["unique_key"])
+ self.assertEqual(
+ {"image_format": "plain"}, build["metadata_override"])
+ self.assertEqual("20140425-103800", build["version"])
+ self.assertIsNone(build["score"])
+ self.assertTrue(build["can_be_rescored"])
+ self.assertFalse(build["can_be_retried"])
+ self.assertFalse(build["can_be_cancelled"])
+
+ def test_public(self):
+ # A LiveFSBuild with a public LiveFS and archive is itself public.
+ db_build = self.factory.makeLiveFSBuild()
+ build_url = api_url(db_build)
+ unpriv_webservice = webservice_for_person(
+ self.factory.makePerson(), permission=OAuthPermission.WRITE_PUBLIC)
+ unpriv_webservice.default_api_version = "devel"
+ logout()
+ self.assertEqual(200, self.webservice.get(build_url).status)
+ self.assertEqual(200, unpriv_webservice.get(build_url).status)
+
+ def test_private_livefs(self):
+ # A LiveFSBuild with a private LiveFS is private.
+ db_team = self.factory.makeTeam(
+ owner=self.person, visibility=PersonVisibility.PRIVATE)
+ with person_logged_in(self.person):
+ db_build = self.factory.makeLiveFSBuild(owner=db_team)
+ build_url = api_url(db_build)
+ unpriv_webservice = webservice_for_person(
+ self.factory.makePerson(), permission=OAuthPermission.WRITE_PUBLIC)
+ unpriv_webservice.default_api_version = "devel"
+ logout()
+ self.assertEqual(200, self.webservice.get(build_url).status)
+ self.assertEqual(401, unpriv_webservice.get(build_url).status)
+
+ def test_private_archive(self):
+ # A LiveFSBuild with a private archive is private.
+ db_archive = self.factory.makeArchive(owner=self.person, private=True)
+ with person_logged_in(self.person):
+ db_build = self.factory.makeLiveFSBuild(archive=db_archive)
+ build_url = api_url(db_build)
+ unpriv_webservice = webservice_for_person(
+ self.factory.makePerson(), permission=OAuthPermission.WRITE_PUBLIC)
+ unpriv_webservice.default_api_version = "devel"
+ logout()
+ self.assertEqual(200, self.webservice.get(build_url).status)
+ self.assertEqual(401, unpriv_webservice.get(build_url).status)
+
+ def test_cancel(self):
+ # The owner of a build can cancel it.
+ db_build = self.factory.makeLiveFSBuild(requester=self.person)
+ db_build.queueBuild()
+ build_url = api_url(db_build)
+ unpriv_webservice = webservice_for_person(
+ self.factory.makePerson(), permission=OAuthPermission.WRITE_PUBLIC)
+ unpriv_webservice.default_api_version = "devel"
+ logout()
+ build = self.webservice.get(build_url).jsonBody()
+ self.assertTrue(build["can_be_cancelled"])
+ response = unpriv_webservice.named_post(build["self_link"], "cancel")
+ self.assertEqual(401, response.status)
+ response = self.webservice.named_post(build["self_link"], "cancel")
+ self.assertEqual(200, response.status)
+ build = self.webservice.get(build_url).jsonBody()
+ self.assertFalse(build["can_be_cancelled"])
+ with person_logged_in(self.person):
+ self.assertEqual(BuildStatus.CANCELLED, db_build.status)
+
+ def test_rescore(self):
+ # Buildd administrators can rescore builds.
+ db_build = self.factory.makeLiveFSBuild(requester=self.person)
+ db_build.queueBuild()
+ build_url = api_url(db_build)
+ buildd_admin = self.factory.makePerson(
+ member_of=[getUtility(ILaunchpadCelebrities).buildd_admin])
+ buildd_admin_webservice = webservice_for_person(
+ buildd_admin, permission=OAuthPermission.WRITE_PUBLIC)
+ buildd_admin_webservice.default_api_version = "devel"
+ logout()
+ build = self.webservice.get(build_url).jsonBody()
+ self.assertEqual(2505, build["score"])
+ self.assertTrue(build["can_be_rescored"])
+ response = self.webservice.named_post(
+ build["self_link"], "rescore", score=5000)
+ self.assertEqual(401, response.status)
+ response = buildd_admin_webservice.named_post(
+ build["self_link"], "rescore", score=5000)
+ self.assertEqual(200, response.status)
+ build = self.webservice.get(build_url).jsonBody()
+ self.assertEqual(5000, build["score"])
+
+ def makeNonRedirectingBrowser(self, person):
+ # The test browser can only work with the appserver, not the
+ # librarian, so follow one layer of redirection through the
+ # appserver and then ask the librarian for the real file.
+ browser = Browser(mech_browser=NonRedirectingMechanizeBrowser())
+ browser.handleErrors = False
+ with person_logged_in(person):
+ browser.addHeader(
+ "Authorization", "Basic %s:test" % person.preferredemail.email)
+ return browser
+
+ def assertCanOpenRedirectedUrl(self, browser, url):
+ redirection = self.assertRaises(HTTPError, browser.open, url)
+ self.assertEqual(303, redirection.code)
+ urlopen(redirection.hdrs["Location"]).close()
+
+ def test_logs(self):
+ # API clients can fetch the build and upload logs.
+ db_build = self.factory.makeLiveFSBuild(requester=self.person)
+ db_build.setLog(self.factory.makeLibraryFileAlias("buildlog.txt.gz"))
+ db_build.storeUploadLog("uploaded")
+ build_url = api_url(db_build)
+ logout()
+ build = self.webservice.get(build_url).jsonBody()
+ browser = self.makeNonRedirectingBrowser(self.person)
+ self.assertIsNotNone(build["build_log_url"])
+ self.assertCanOpenRedirectedUrl(browser, build["build_log_url"])
+ self.assertIsNotNone(build["upload_log_url"])
+ self.assertCanOpenRedirectedUrl(browser, build["upload_log_url"])
+
+ def test_getFileUrls(self):
+ # API clients can fetch files attached to builds.
+ db_build = self.factory.makeLiveFSBuild(requester=self.person)
+ db_files = [
+ self.factory.makeLiveFSFile(livefsbuild=db_build)
+ for i in range(2)]
+ build_url = api_url(db_build)
+ file_urls = [
+ ProxiedLibraryFileAlias(file.libraryfile, db_build).http_url
+ for file in db_files]
+ logout()
+ response = self.webservice.named_get(build_url, "getFileUrls")
+ self.assertEqual(200, response.status)
+ self.assertContentEqual(file_urls, response.jsonBody())
+ browser = self.makeNonRedirectingBrowser(self.person)
+ for file_url in file_urls:
+ self.assertCanOpenRedirectedUrl(browser, file_url)
=== added file 'lib/lp/soyuz/tests/test_livefsbuildbehaviour.py'
--- lib/lp/soyuz/tests/test_livefsbuildbehaviour.py 1970-01-01 00:00:00 +0000
+++ lib/lp/soyuz/tests/test_livefsbuildbehaviour.py 2014-04-29 17:27:20 +0000
@@ -0,0 +1,213 @@
+# Copyright 2014 Canonical Ltd. This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test live filesystem build behaviour."""
+
+__metaclass__ = type
+
+from datetime import datetime
+from textwrap import dedent
+
+import fixtures
+import pytz
+from testtools import run_test_with
+from testtools.deferredruntest import (
+ assert_fails_with,
+ AsynchronousDeferredRunTest,
+ )
+import transaction
+from twisted.internet import defer
+from twisted.trial.unittest import TestCase as TrialTestCase
+from zope.component import getUtility
+from zope.security.proxy import removeSecurityProxy
+
+from lp.buildmaster.enums import BuildStatus
+from lp.buildmaster.interfaces.builder import CannotBuild
+from lp.buildmaster.interfaces.buildfarmjobbehaviour import (
+ IBuildFarmJobBehaviour,
+ )
+from lp.buildmaster.tests.mock_slaves import (
+ MockBuilder,
+ OkSlave,
+ )
+from lp.buildmaster.tests.test_buildfarmjobbehaviour import (
+ TestGetUploadMethodsMixin,
+ TestHandleStatusMixin,
+ )
+from lp.registry.interfaces.pocket import PackagePublishingPocket
+from lp.services.features.testing import FeatureFixture
+from lp.services.log.logger import BufferLogger
+from lp.soyuz.adapters.archivedependencies import get_sources_list_for_building
+from lp.soyuz.interfaces.livefs import LIVEFS_FEATURE_FLAG
+from lp.soyuz.interfaces.processor import IProcessorSet
+from lp.soyuz.model.livefsbuildbehaviour import LiveFSBuildBehaviour
+from lp.testing import TestCaseWithFactory
+from lp.testing.layers import LaunchpadZopelessLayer
+
+
+class TestLiveFSBuildBehaviour(TestCaseWithFactory):
+
+ layer = LaunchpadZopelessLayer
+
+ def setUp(self):
+ super(TestLiveFSBuildBehaviour, self).setUp()
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+
+ def makeJob(self, **kwargs):
+ """Create a sample `ILiveFSBuildBehaviour`."""
+ distribution = self.factory.makeDistribution(name="distro")
+ distroseries = self.factory.makeDistroSeries(
+ distribution=distribution, name="unstable")
+ processor = getUtility(IProcessorSet).getByName("386")
+ distroarchseries = self.factory.makeDistroArchSeries(
+ distroseries=distroseries, architecturetag="i386",
+ processor=processor)
+ build = self.factory.makeLiveFSBuild(
+ distroarchseries=distroarchseries,
+ pocket=PackagePublishingPocket.RELEASE, name=u"livefs", **kwargs)
+ return IBuildFarmJobBehaviour(build)
+
+ def test_provides_interface(self):
+ # LiveFSBuildBehaviour provides IBuildFarmJobBehaviour.
+ job = LiveFSBuildBehaviour(None)
+ self.assertProvides(job, IBuildFarmJobBehaviour)
+
+ def test_adapts_ILiveFSBuild(self):
+ # IBuildFarmJobBehaviour adapts an ILiveFSBuild.
+ build = self.factory.makeLiveFSBuild()
+ job = IBuildFarmJobBehaviour(build)
+ self.assertProvides(job, IBuildFarmJobBehaviour)
+
+ def test_displayname(self):
+ # displayname contains a reasonable description of the job.
+ job = self.makeJob()
+ self.assertEqual(
+ "i386 build of livefs in distro unstable RELEASE", job.displayname)
+
+ def test_logStartBuild(self):
+ # logStartBuild will properly report the image that's being built.
+ job = self.makeJob()
+ logger = BufferLogger()
+ job.logStartBuild(logger)
+ self.assertEqual(
+ "INFO startBuild(i386 build of livefs in distro unstable "
+ "RELEASE)\n", logger.getLogBuffer())
+
+ def test_verifyBuildRequest_valid(self):
+ # verifyBuildRequest doesn't raise any exceptions when called with a
+ # valid builder set.
+ job = self.makeJob()
+ lfa = self.factory.makeLibraryFileAlias()
+ transaction.commit()
+ job.build.distroarchseries.addOrUpdateChroot(lfa)
+ builder = MockBuilder()
+ job.setBuilder(builder, OkSlave())
+ logger = BufferLogger()
+ job.verifyBuildRequest(logger)
+ self.assertEqual("", logger.getLogBuffer())
+
+ def test_verifyBuildRequest_virtual_mismatch(self):
+ # verifyBuildRequest raises on an attempt to build a virtualized
+ # build on a non-virtual builder.
+ job = self.makeJob()
+ lfa = self.factory.makeLibraryFileAlias()
+ transaction.commit()
+ job.build.distroarchseries.addOrUpdateChroot(lfa)
+ builder = MockBuilder(virtualized=False)
+ job.setBuilder(builder, OkSlave())
+ logger = BufferLogger()
+ e = self.assertRaises(AssertionError, job.verifyBuildRequest, logger)
+ self.assertEqual(
+ "Attempt to build virtual item on a non-virtual builder.", str(e))
+
+ def test_verifyBuildRequest_no_chroot(self):
+ # verifyBuildRequest raises when the DAS has no chroot.
+ job = self.makeJob()
+ builder = MockBuilder()
+ job.setBuilder(builder, OkSlave())
+ logger = BufferLogger()
+ e = self.assertRaises(CannotBuild, job.verifyBuildRequest, logger)
+ self.assertIn("Missing chroot", str(e))
+
+ def test_getBuildCookie(self):
+ # A build cookie is made up of the job type and record id. The
+ # uploadprocessor relies on this format.
+ job = self.makeJob()
+ cookie = removeSecurityProxy(job).getBuildCookie()
+ self.assertEqual("LIVEFSBUILD-%s" % job.build.id, cookie)
+
+ def test_extraBuildArgs(self):
+ # _extraBuildArgs returns a reasonable set of additional arguments.
+ job = self.makeJob(
+ date_created=datetime(2014, 04, 25, 10, 38, 0, tzinfo=pytz.UTC),
+ metadata={"project": "distro", "subproject": "special"})
+ expected_archives = get_sources_list_for_building(
+ job.build, job.build.distroarchseries, None)
+ self.assertEqual({
+ "archive_private": False,
+ "archives": expected_archives,
+ "arch_tag": "i386",
+ "datestamp": "20140425-103800",
+ "project": "distro",
+ "subproject": "special",
+ "suite": "unstable",
+ }, job._extraBuildArgs())
+
+ @run_test_with(AsynchronousDeferredRunTest)
+ @defer.inlineCallbacks
+ def test_dispatchBuildToSlave(self):
+ # dispatchBuildToSlave makes the proper calls to the slave.
+ job = self.makeJob()
+ lfa = self.factory.makeLibraryFileAlias()
+ transaction.commit()
+ job.build.distroarchseries.addOrUpdateChroot(lfa)
+ slave = OkSlave()
+ builder = MockBuilder("bob")
+ builder.processor = getUtility(IProcessorSet).getByName("386")
+ job.setBuilder(builder, slave)
+ logger = BufferLogger()
+ yield job.dispatchBuildToSlave("someid", logger)
+ self.assertStartsWith(
+ logger.getLogBuffer(),
+ dedent("""\
+ INFO Sending chroot file for live filesystem build to bob
+ INFO Initiating build 1-someid on http://fake:0000
+ """))
+ self.assertEqual(
+ ["ensurepresent", "build"], [call[0] for call in slave.call_log])
+ build_args = slave.call_log[1][1:]
+ self.assertEqual(job.getBuildCookie(), build_args[0])
+ self.assertEqual("livefs", build_args[1])
+ self.assertEqual([], build_args[3])
+ self.assertEqual(job._extraBuildArgs(), build_args[4])
+
+ @run_test_with(AsynchronousDeferredRunTest)
+ def test_dispatchBuildToSlave_no_chroot(self):
+ # dispatchBuildToSlave fails when the DAS has no chroot.
+ job = self.makeJob()
+ builder = MockBuilder()
+ builder.processor = getUtility(IProcessorSet).getByName("386")
+ job.setBuilder(builder, OkSlave())
+ d = job.dispatchBuildToSlave("someid", BufferLogger())
+ return assert_fails_with(d, CannotBuild)
+
+
+class MakeLiveFSBuildMixin:
+ """Provide the common makeBuild method returning a queued build."""
+
+ def makeBuild(self):
+ self.useFixture(FeatureFixture({LIVEFS_FEATURE_FLAG: u"on"}))
+ build = self.factory.makeLiveFSBuild(status=BuildStatus.BUILDING)
+ build.queueBuild()
+ return build
+
+
+class TestGetUploadMethodsForLiveFSBuild(
+ MakeLiveFSBuildMixin, TestGetUploadMethodsMixin, TestCaseWithFactory):
+ """IPackageBuild.getUpload-related methods work with LiveFS builds."""
+
+
+class TestHandleStatusForLiveFSBuild(
+ MakeLiveFSBuildMixin, TestHandleStatusMixin, TrialTestCase,
+ fixtures.TestWithFixtures):
+ """IPackageBuild.handleStatus works with LiveFS builds."""
=== modified file 'lib/lp/testing/factory.py'
--- lib/lp/testing/factory.py 2014-04-24 02:16:27 +0000
+++ lib/lp/testing/factory.py 2014-04-29 17:27:20 +0000
@@ -2,7 +2,7 @@
# NOTE: The first line above must stay first; do not move the copyright
# notice to the top. See http://www.python.org/dev/peps/pep-0263/.
#
-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
"""Testing infrastructure for the Launchpad application.
@@ -283,6 +283,8 @@
IComponent,
IComponentSet,
)
+from lp.soyuz.interfaces.livefs import ILiveFSSet
+from lp.soyuz.interfaces.livefsbuild import ILiveFSBuildSet
from lp.soyuz.interfaces.packagecopyjob import IPlainPackageCopyJobSource
from lp.soyuz.interfaces.packageset import IPackagesetSet
from lp.soyuz.interfaces.processor import IProcessorSet
@@ -297,6 +299,7 @@
BinaryPackageFile,
SourcePackageReleaseFile,
)
+from lp.soyuz.model.livefsbuild import LiveFSFile
from lp.soyuz.model.packagediff import PackageDiff
from lp.testing import (
admin_logged_in,
@@ -4316,6 +4319,77 @@
product.redeemSubscriptionVoucher(
self.getUniqueString(), person, person, months)
+ def makeLiveFS(self, registrant=None, owner=None, distroseries=None,
+ name=None, metadata=None, date_created=DEFAULT):
+ """Make a new LiveFS."""
+ if registrant is None:
+ registrant = self.makePerson()
+ if owner is None:
+ owner = self.makePerson()
+ if distroseries is None:
+ distroseries = self.makeDistroSeries()
+ if name is None:
+ name = self.getUniqueString(u"livefs-name")
+ if metadata is None:
+ metadata = {}
+ livefs = getUtility(ILiveFSSet).new(
+ registrant, owner, distroseries, name, metadata,
+ date_created=date_created)
+ IStore(livefs).flush()
+ return livefs
+
+ def makeLiveFSBuild(self, requester=None, livefs=None, archive=None,
+ distroarchseries=None, pocket=None, unique_key=None,
+ metadata_override=None, date_created=DEFAULT,
+ status=BuildStatus.NEEDSBUILD, builder=None,
+ duration=None, **kwargs):
+ """Make a new LiveFSBuild."""
+ if livefs is None:
+ if "distroseries" in kwargs:
+ distroseries = kwargs["distroseries"]
+ del kwargs["distroseries"]
+ elif distroarchseries is not None:
+ distroseries = distroarchseries.distroseries
+ elif archive is not None:
+ distroseries = self.makeDistroSeries(
+ distribution=archive.distribution)
+ else:
+ distroseries = None
+ livefs = self.makeLiveFS(distroseries=distroseries, **kwargs)
+ if requester is None:
+ requester = self.makePerson()
+ if archive is None:
+ archive = livefs.distroseries.main_archive
+ if distroarchseries is None:
+ distroarchseries = self.makeDistroArchSeries(
+ distroseries=livefs.distroseries)
+ if pocket is None:
+ pocket = PackagePublishingPocket.RELEASE
+ livefsbuild = getUtility(ILiveFSBuildSet).new(
+ requester, livefs, archive, distroarchseries, pocket,
+ unique_key=unique_key, metadata_override=metadata_override,
+ date_created=date_created)
+ if duration is not None:
+ removeSecurityProxy(livefsbuild).updateStatus(
+ BuildStatus.BUILDING, builder=builder,
+ date_started=livefsbuild.date_created)
+ removeSecurityProxy(livefsbuild).updateStatus(
+ status, builder=builder,
+ date_finished=livefsbuild.date_started + duration)
+ else:
+ removeSecurityProxy(livefsbuild).updateStatus(
+ status, builder=builder)
+ IStore(livefsbuild).flush()
+ return livefsbuild
+
+ def makeLiveFSFile(self, livefsbuild=None, libraryfile=None):
+ if livefsbuild is None:
+ livefsbuild = self.makeLiveFSBuild()
+ if libraryfile is None:
+ libraryfile = self.makeLibraryFileAlias()
+ return ProxyFactory(
+ LiveFSFile(livefsbuild=livefsbuild, libraryfile=libraryfile))
+
# Some factory methods return simple Python types. We don't add
# security wrappers for them, as well as for objects created by
Follow ups
