launchpad-reviewers team mailing list archive

[Steve Kowalik <stevenk@xxxxxxxxxx>]

Steve Kowalik has proposed merging lp:~stevenk/launchpad/getpublishedsources-too-eager into lp:launchpad.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  Bug #744849 in Launchpad itself: "populate-archive.py unusably slow"
  https://bugs.launchpad.net/launchpad/+bug/744849

For more details, see:
https://code.launchpad.net/~stevenk/launchpad/getpublishedsources-too-eager/+merge/57107

Add a eager_load parameter to IArchive.getPublishedSources() that disables eager loading for those callsites that do not require it. It defaults to True, so the API/page renders will always get it. So far the only callsite that disables it is the package cloner.
-- 
https://code.launchpad.net/~stevenk/launchpad/getpublishedsources-too-eager/+merge/57107
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~stevenk/launchpad/getpublishedsources-too-eager into lp:launchpad.

=== modified file 'lib/lp/soyuz/interfaces/archive.py'
--- lib/lp/soyuz/interfaces/archive.py	2011-02-24 15:30:54 +0000
+++ lib/lp/soyuz/interfaces/archive.py	2011-04-11 04:56:30 +0000
@@ -991,7 +991,8 @@
     @export_read_operation()
     def getPublishedSources(name=None, version=None, status=None,
                             distroseries=None, pocket=None,
-                            exact_match=False, created_since_date=None):
+                            exact_match=False, created_since_date=None,
+                            eager_load=True):
         """All `ISourcePackagePublishingHistory` target to this archive.
 
         :param name: source name filter (exact match or SQL LIKE controlled

=== modified file 'lib/lp/soyuz/model/archive.py'
--- lib/lp/soyuz/model/archive.py	2011-03-23 18:29:09 +0000
+++ lib/lp/soyuz/model/archive.py	2011-04-11 04:56:30 +0000
@@ -465,7 +465,8 @@
 
     def getPublishedSources(self, name=None, version=None, status=None,
                             distroseries=None, pocket=None,
-                            exact_match=False, created_since_date=None):
+                            exact_match=False, created_since_date=None,
+                            eager_load=True):
         """See `IArchive`."""
         # clauses contains literal sql expressions for things that don't work
         # easily in storm : this method was migrated from sqlobject but some
@@ -526,6 +527,8 @@
         resultset = store.find(SourcePackagePublishingHistory,
             *storm_clauses).order_by(
             *orderBy)
+        if eager_load is False:
+            return resultset
         # Its not clear that this eager load is necessary or sufficient, it
         # replaces a prejoin that had pathological query plans.
         def eager_load(rows):

=== modified file 'lib/lp/soyuz/model/packagecloner.py'
--- lib/lp/soyuz/model/packagecloner.py	2011-03-03 00:43:44 +0000
+++ lib/lp/soyuz/model/packagecloner.py	2011-04-11 04:56:30 +0000
@@ -133,7 +133,8 @@
         # as PUBLISHED. It's part of the assumptions made in:
         # https://launchpad.net/soyuz/+spec/build-unpublished-source
         sources_published = archive.getPublishedSources(
-            distroseries=distroseries, status=active_publishing_status)
+            distroseries=distroseries, status=active_publishing_status,
+            eager_load=False)
 
         for pubrec in sources_published:
             builds = pubrec.createMissingBuilds(