launchpad-reviewers team mailing list archive

[Steve Kowalik <stevenk@xxxxxxxxxx>]

Steve Kowalik has proposed merging lp:~stevenk/launchpad/branch-ap-watermark into lp:launchpad.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~stevenk/launchpad/branch-ap-watermark/+merge/111322

Sort out issues discovered with the PopulateBranchAccessPolicy garbo job on qastaging. First, a watermark is now used so we trundle over the branches once. Second, we now call the branch_denorm_access() trigger as well to correctly deal with branches which had APAs created by model code, and so _reconcileAccess() has no effect.
-- 
https://code.launchpad.net/~stevenk/launchpad/branch-ap-watermark/+merge/111322
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~stevenk/launchpad/branch-ap-watermark into lp:launchpad.

=== modified file 'lib/lp/scripts/garbo.py'
--- lib/lp/scripts/garbo.py	2012-06-19 05:22:50 +0000
+++ lib/lp/scripts/garbo.py	2012-06-21 00:11:20 +0000
@@ -1143,11 +1143,18 @@
 
     maximum_chunk_size = 5000
 
+    def __init__(self, log, abort_time=None):
+        super(PopulateBranchAccessPolicy, self).__init__(log, abort_time)
+        self.memcache_key = '%s:branch-populate-ap' % config.instance_name
+        watermark = getUtility(IMemcacheClient).get(self.memcache_key)
+        self.start_at = watermark or 0
+
     def findBranches(self):
         return IMasterStore(Branch).find(
             Branch,
             Branch.information_type.is_in(PRIVATE_INFORMATION_TYPES),
-            SQL("Branch.access_policy IS NULL"))
+            SQL("Branch.access_policy IS NULL"),
+            Branch.id >= self.start_at).order_by(Branch.id)
 
     def isDone(self):
         return self.findBranches().is_empty()
@@ -1155,6 +1162,13 @@
     def __call__(self, chunk_size):
         for branch in self.findBranches()[:chunk_size]:
             branch._reconcileAccess()
+            IMasterStore(Branch).execute(
+                'SELECT branch_denorm_access(?)', branch.id)
+            self.start_at = branch.id + 1
+        result = getUtility(IMemcacheClient).set(
+            self.memcache_key, self.start_at)
+        if not result:
+            self.log.warning('Failed to set start_at in memcache.')
         transaction.commit()