launchpad-dev team mailing list archive

[William Grant <william.grant@xxxxxxxxxxxxx>]

On 31/03/12 00:59, Stuart Bishop wrote:
> The staging databases have been upgraded to PostgreSQL 9.1.
> 
> The upgrade was done inplace, taking just under 10 minutes for
> pg_upgrade to run. Actual downtime was longer as we had missed one of
> the preparatory steps (checklists updated).
> 
> Clients running under our standard Lucid environment fail to
> communicate sanely with PostgreSQL 9.1, as text types get quoted
> incorrectly. The problem and fix was immediately apparent because
> Landscape had suffered the exact same issue. I've opened RT#51902, as
> it is now apparent that all our servers running PostgreSQL clients
> need to be upgraded from libpq4 to libpq5.

Note that "fail to communicate sanely" means "get corrupt strings", not
"crash". Unless further checking is performed on received strings, it's
unlikely that anyone will notice an upgrade has been missed until
corruption spreads elsewhere.

> Apart from smoke testing on staging, including cross version
> replication, we are ready for production upgrades to start 23rd April.
> The Precise release schedule will of course be the biggest factor in
> deciding when the upgrades actually start. Upgrades will be staggered,
> slave databases first. I expect the upgrade of the master will be done
> with a scheduled 30 minute outage window. The alternative is two short
> outages while we promote slaves to master and a period we are running
> with a single slave and half our usual CPU power on the master.

A 30 minute outage is extremely disruptive and unprecedented since
fastdowntime, so I'd really prefer that we didn't. With recent
optimisations we can sensibly run the master on a slave's hardware for
even an extended period, as long as we first tweak the replication code
to use a slave if it's up to date, rather than only using a slave if the
entire cluster is up to date.

Attachment: signature.asc
Description: OpenPGP digital signature