launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #07233
[Merge] lp:~stub/launchpad/staging into lp:launchpad
Stuart Bishop has proposed merging lp:~stub/launchpad/staging into lp:launchpad.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~stub/launchpad/staging/+merge/103295
= Summary =
The staging database server doesn't have enough space to support our current staging update process. We build a fresh staging database, replicate it, and swap the databases into place. At the high point, this requires storing 4 complete copies of the production database.
== Proposed fix ==
Swap the staging databases into place without waiting for the slave database to complete being built. This will reduce the disk foot print to 3 complete copies of the production database.
== Pre-implementation notes ==
== LOC Rationale ==
== Implementation details ==
== Tests ==
== Demo and Q/A ==
= Launchpad lint =
Checking for conflicts and issues in changed files.
Linting changed files:
database/replication/Makefile
database/replication/new-slave.py
./database/replication/new-slave.py
11: '_pythonpath' imported but unused
--
https://code.launchpad.net/~stub/launchpad/staging/+merge/103295
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~stub/launchpad/staging into lp:launchpad.
=== modified file 'database/replication/Makefile'
--- database/replication/Makefile 2012-04-02 12:00:50 +0000
+++ database/replication/Makefile 2012-04-24 14:29:58 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
#
# This makefile is used for two task:
@@ -135,6 +135,21 @@
# irrelevant, as if our dump was created with --exclude-schema=_sl
# it will not contain anything to remove.
LPCONFIG=${NEW_STAGING_CONFIG} ./repair-restored-db.py
+ # Apply database patches now. This is unfortunate, as it means
+ # the patches are not being applied to a replicated database.
+ # With this update, we get incorrect patch application times and
+ # won't catch patches incompatible with Slony. We do this now,
+ # because we are running short of disk space on the staging
+ # database server. With the patches being applied before
+ # replication, we can swap the slave database into place before
+ # it has finished building and we end up using only 3*production
+ # amount of disk space rather than 4*production.
+ @echo Running upgrade.py `date` non-replicated. Times need adjustment.
+ LPCONFIG=${NEW_STAGING_CONFIG} ${SHHH} ../schema/upgrade.py \
+ --log-file=INFO:${STAGING_LOGDIR}/dbupgrade.log
+ @echo Running security.py `date`
+ LPCONFIG=${NEW_STAGING_CONFIG} ${SHHH} ../schema/security.py -U slony \
+ --log-file=INFO:${STAGING_LOGDIR}/dbupgrade.log
# Setup replication
make _replicate LPCONFIG=${NEW_STAGING_CONFIG} LAG="0 seconds" \
_MASTER=lpmain_staging_new _SLAVE=lpmain_staging_slave_new \
@@ -203,15 +218,8 @@
# Setup the slave
./new-slave.py 2 "dbname=${_SLAVE}" \
--log-file=INFO:${STAGING_LOGDIR}/dbupgrade.log
- # Upgrade all databases in the cluster and reset security.
- @echo Running upgrade.py `date`
- ${SHHH} ../schema/upgrade.py \
- --log-file=INFO:${STAGING_LOGDIR}/dbupgrade.log
- @echo Running security.py `date`
- ./slon_ctl.py stop # security.py can deadlock with slony
- ${SHHH} ../schema/security.py --cluster -U slony \
- --log-file=INFO:${STAGING_LOGDIR}/dbupgrade.log
# Restart slon daemons with default lag setting.
+ ./slon_ctl.py stop
./slon_ctl.py --lag="${LAG}" start
# Generate a preamble for manual slonik(1) usage.
./preamble.py > preamble.sk
=== modified file 'database/replication/new-slave.py'
--- database/replication/new-slave.py 2011-12-30 06:47:54 +0000
+++ database/replication/new-slave.py 2012-04-24 14:29:58 +0000
@@ -121,12 +121,10 @@
"Database at %s is not empty." % target_connection_string)
target_con.rollback()
- # Duplicate the full schema. We restore with no-privileges as required
- # roles may not yet exist, so we have to run security.py on the
- # new slave once it is built.
+ # Duplicate the full schema.
log.info("Duplicating full db schema from '%s' to '%s'" % (
lpmain_connection_string, target_connection_string))
- cmd = "pg_dump --schema-only --no-privileges %s | psql -1 -q %s" % (
+ cmd = "pg_dump --schema-only %s | psql -1 -q %s" % (
source_connection_string.asPGCommandLineArgs(),
target_connection_string.asPGCommandLineArgs())
if subprocess.call(cmd, shell=True) != 0:
@@ -219,13 +217,10 @@
subscribe set (
id=%d, provider=@master_node, receiver=@new_node, forward=yes);
echo 'Waiting for subscribe to start processing.';
- echo 'This will block on long running transactions.';
- sync (id = @master_node);
wait for event (
origin = @master_node, confirmed = ALL,
wait on = @master_node, timeout = 0);
""" % (set_id, set_id))
- script += full_sync
replication.helpers.execute_slonik(script)
Follow ups