launchpad-reviewers team mailing list archive

[Colin Watson <mp+426627@xxxxxxxxxxxxxxxxxx>]

Colin Watson has proposed merging ~cjwatson/launchpad:db-upgrade-separate-sessions into launchpad:db-devel.

Commit message:
Have "make schema" apply DB patches in separate sessions

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)

For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/426627

This is temporarily necessary because of the PL/Python 2 to 3 migration: PostgreSQL refuses to have both old and new functions referenced in the same session, which happens if we try to apply the full 2210-* patch sequence at once.  It's a bit awkward because there's no obvious way to make `--dry-run` work in conjunction with this mode, but we should be able to revert this after rebaselining.
-- 
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:db-upgrade-separate-sessions into launchpad:db-devel.

diff --git a/database/schema/Makefile b/database/schema/Makefile
index 65ac73a..5d303fb 100644
--- a/database/schema/Makefile
+++ b/database/schema/Makefile
@@ -115,7 +115,7 @@ create:
 	@ echo "* Loading base database schema"
 	@ psql -d ${EMPTY_DBNAME} -f ${BASELINE} | grep : | cat
 	@ echo "* Patching the database schema"
-	@ ./upgrade.py -d ${EMPTY_DBNAME}
+	@ ./upgrade.py --separate-sessions -d ${EMPTY_DBNAME}
 	@ echo "* Security setup"
 	@ ./security.py -q -d ${EMPTY_DBNAME}
 	@ echo "* Disabling autovacuum"
diff --git a/database/schema/upgrade.py b/database/schema/upgrade.py
index 3b579ff..16fcf12 100755
--- a/database/schema/upgrade.py
+++ b/database/schema/upgrade.py
@@ -37,7 +37,11 @@ def main(con=None):
     patches = get_patchlist(con)
 
     log.info("Applying patches.")
-    apply_patches_normal(con)
+    if options.separate_sessions:
+        apply_patches_separately()
+        con.rollback()
+    else:
+        apply_patches_normal(con)
 
     report_patch_times(con, patches)
 
@@ -163,6 +167,26 @@ def apply_patches_normal(con):
     apply_comments(con)
 
 
+def apply_patches_separately():
+    """Update a database, applying each patch in a separate session."""
+    # Apply the patches.
+    with connect() as con:
+        patches = get_patchlist(con)
+    for (major, minor, patch), patch_file in patches:
+        with connect() as con:
+            apply_patch(con, major, minor, patch, patch_file)
+
+    # Update comments.
+    with connect() as con:
+        apply_comments(con)
+
+    # Commit changes.  (Not optional, since we don't currently support using
+    # --dry-run and --separate-sessions together; it's not clear how this
+    # would work when applying multiple patches.)
+    log.debug("Committing changes")
+    con.commit()
+
+
 def get_patchlist(con):
     """Return a patches that need to be applied to the connected database
     in [((major, minor, patch), patch_file)] format.
@@ -276,10 +300,15 @@ if __name__ == '__main__':
     parser.add_option(
         "--skip-comments", dest="comments", default=True,
         action="store_false", help="Skip applying comments.sql")
+    parser.add_option(
+        "--separate-sessions", dest="separate_sessions", default=False,
+        action="store_true", help="Apply each patch in a separate session")
     (options, args) = parser.parse_args()
 
     if args:
         parser.error("Too many arguments")
+    if not options.commit and options.separate_sessions:
+        parser.error("--dry-run and --separate-sessions are incompatible")
 
     log = logger(options)
     main()