launchpad-reviewers team mailing list archive

[Stuart Bishop <stuart.bishop@xxxxxxxxxxxxx>]

Stuart Bishop has proposed merging lp:~stub/launchpad/distinct-db-users into lp:launchpad with lp:~stub/launchpad/trivial as a prerequisite.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  Bug #815753 in Launchpad itself: "process-accepted, publish-distro, process-death-row, buildd-retry-depwait and generate-contents-files should use their own DB users"
  https://bugs.launchpad.net/launchpad/+bug/815753

For more details, see:
https://code.launchpad.net/~stub/launchpad/distinct-db-users/+merge/76535

= Summary =

fastdowntime db updates are not as smooth as they could be, as we need to shut down too much. This is because many processes are sharing database users, and we cannot distinguish the robust connections from the genuinely fragile connections.

== Proposed fix ==

Fix the fragile systems to connect as distinct database users.

== Pre-implementation notes ==

== Implementation details ==

== Tests ==

== Demo and Q/A ==


= Launchpad lint =

(Fixing now)
-- 
https://code.launchpad.net/~stub/launchpad/distinct-db-users/+merge/76535
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~stub/launchpad/distinct-db-users into lp:launchpad.

=== modified file 'cronscripts/buildd-retry-depwait.py'
--- cronscripts/buildd-retry-depwait.py	2010-04-27 19:48:39 +0000
+++ cronscripts/buildd-retry-depwait.py	2011-09-22 09:47:34 +0000
@@ -28,8 +28,7 @@
 from lp.soyuz.scripts.buildd import RetryDepwait
 
 if __name__ == '__main__':
-    script = RetryDepwait(
-        'retry-depwait', dbuser=config.builddmaster.dbuser)
+    script = RetryDepwait('retry-depwait', dbuser='retry_depwait')
     script.lock_or_quit()
     try:
         script.run()

=== modified file 'database/schema/preflight.py'
--- database/schema/preflight.py	2011-09-16 14:55:43 +0000
+++ database/schema/preflight.py	2011-09-22 09:47:34 +0000
@@ -40,7 +40,14 @@
 # added here. The preflight check will fail if any of these users are
 # connected, so these systems will need to be shut down manually before
 # a database update.
-FRAGILE_USERS = frozenset(['archivepublisher', 'buildd_manager'])
+FRAGILE_USERS = frozenset([
+    'buildd_manager',
+    # process_accepted is fragile, but also fast so we likely shouldn't
+    # need to ever manually shut it down.
+    'process_accepted',
+    'process_upload',
+    'publish_distro',
+    ])
 
 # How lagged the cluster can be before failing the preflight check.
 MAX_LAG = timedelta(seconds=60)

=== modified file 'database/schema/security.cfg'
--- database/schema/security.cfg	2011-09-22 01:53:49 +0000
+++ database/schema/security.cfg	2011-09-22 09:47:34 +0000
@@ -927,6 +927,10 @@
 type=user
 groups=fiera
 
+[retry_depwait]
+type=user
+groups=fiera
+
 [ppa-apache-log-parser]
 groups=script
 public.archive                                  = SELECT
@@ -1212,6 +1216,10 @@
 groups=script,uploading
 type=user
 
+[process_upload]
+groups=uploader
+type=user
+
 [uploading]
 public.account                          = SELECT, INSERT
 public.accountpassword                  = SELECT, INSERT
@@ -1420,6 +1428,10 @@
 public.validpersonorteamcache           = SELECT
 type=user
 
+[process_accepted]
+type=user
+groups=queued
+
 [ppad]
 groups=script
 public.archive                          = SELECT
@@ -2228,6 +2240,14 @@
 type=user
 groups=archivepublisher
 
+[process_death_row]
+type=user
+groups=archivepublisher
+
+[publish_distro]
+type=user
+groups=archivepublisher
+
 [publish_ftpmaster]
 type=user
 groups=archivepublisher

=== modified file 'scripts/process-accepted.py'
--- scripts/process-accepted.py	2010-04-27 19:48:39 +0000
+++ scripts/process-accepted.py	2011-09-22 09:47:34 +0000
@@ -22,6 +22,6 @@
 
 if __name__ == '__main__':
     script = ProcessAccepted(
-        "process-accepted", dbuser=config.uploadqueue.dbuser)
+        "process-accepted", dbuser='process_accepted')
     script.lock_and_run(isolation=ISOLATION_LEVEL_READ_COMMITTED)
 

=== modified file 'scripts/process-death-row.py'
--- scripts/process-death-row.py	2010-08-25 12:57:29 +0000
+++ scripts/process-death-row.py	2011-09-22 09:47:34 +0000
@@ -37,6 +37,6 @@
 
 if __name__ == "__main__":
     script = DeathRowProcessor(
-        'process-death-row', dbuser=config.archivepublisher.dbuser)
+        'process-death-row', dbuser='process_death_row')
     script.lock_and_run()
 

=== modified file 'scripts/process-upload.py'
--- scripts/process-upload.py	2010-06-23 15:54:45 +0000
+++ scripts/process-upload.py	2011-09-22 09:47:34 +0000
@@ -16,5 +16,5 @@
 
 
 if __name__ == '__main__':
-    script = ProcessUpload('process-upload', dbuser=config.uploader.dbuser)
+    script = ProcessUpload('process-upload', dbuser='process_upload')
     script.lock_and_run()

=== modified file 'scripts/publish-distro.py'
--- scripts/publish-distro.py	2011-08-08 07:31:08 +0000
+++ scripts/publish-distro.py	2011-09-22 09:47:34 +0000
@@ -12,5 +12,5 @@
 
 
 if __name__ == "__main__":
-    script = PublishDistro(dbuser=config.archivepublisher.dbuser)
+    script = PublishDistro('publish-distro', dbuser='publish_distro')
     script.lock_and_run()