openstack team mailing list archive

[Adam Gandelman <adamg@xxxxxxxxxxxxx>]

Hi-

We've added a patch to the Glance package in Ubuntu to help improve the 
experience for users who are counting on a smooth upgrade from Diablo.  
Since the release of Essex, I've been mostly focused on upgrade 
testing.  Things seem generally okay with the exception of Glance and 
the way it manages its database and migrations.  To ensure users can 
reliably upgrade from Diablo, we've made a couple of changes that people 
should know about as the end result is currently a bit different than 
what you'd expect comparing to devstack/vanilla git repos.

1. Disabled database auto-creation

Glance is the only project that I know of that currently allows 
sqlalchemy to auto-create database schema to match its database 
models.   This is great for the developer use-case, but really not ideal 
for anyone who wishes to maintain their database for the long-term.  The 
original issue is that, on startup, glance-registry creates a database 
schema that reflects its most recent view of the database.  When it 
does, the schema is created outside the view of migrate and the 
migration repository cannot be used to incrementally upgrade/downgrade 
the schema.

Recent changes [1] in Glance provides a work-around to this isssue, 
where we rely on glance-registry to automatically put the database under 
version control after auto-creation and set its version to the most 
recent (currently 13).  This works fine for development but there are 
important details in the migration code that don't actually end up in 
the database schema when auto-created (FK constraints, for example) 
because the migration scripts are not actually run.

To avoid this entirely, we've prevented Glance from auto-migrating its 
database and now require users to migrate the database on new install 
and upgrade (via db_sync) in the same way Nova does.

2. Sync'ing a new database.

The changes also now allow users to set a specific version when 
initializing version_control on a database with glance-manage, eg:

glance-manage version_control 9
glance-manage db_sync

This would properly migrate an non-version controlled Diablo database to 
Essex. This is great, but it is still possible for Diablo users to 
mistakenly start glance or run db_sync with no version control, and end 
up in a situation where glance + migrate think it is up to date. In 
reality the database is missing 3 migrations (one of which, 012 IDs to 
UUIDs, is very essential!)   To prevent this, the Ubuntu patch 
additionally requires explicitly version_control'ing the database a 
before db_sync on both new databases and during upgrades of databases 
that are not already version_control'd.

For example, on a new database:

# Start from the beginning of the migration repositroy
glance-manage version_control 0
glance-manage db_sync

or to upgrade from Diablo:

# Was our database properly version controlled to begin with?
if ! glance-manage db_version ; then
  # Set its version if not
  glance-manage version_control 9
fi
# Run all migrations 10 to 13
glance-manage db_sync

I understand this may have come a bit late to any users of the Precise 
packages and I apologize for the lack of a heads up.  I've filed a new 
bug [2] about the split brain between glance-registry and and hope to 
address this at the database management session [3] at ODS next week.  
Anne Gentle has also proposed an update to the documentation with a note 
about this.   Please understand that our aim was to keep our patch delta 
as minimal as possible in Ubuntu, but with the yesterday's Final Freeze 
for the cycle 12.04 we needed something in place that ensures a sane 
user story for previous users of Diablo and future Essex users alike.

Cheers,
Adam

[1] https://review.openstack.org/#/c/6166/ 
https://review.openstack.org/#/c/5980/1
[2] https://bugs.launchpad.net/bugs/981081
[3] http://summit.openstack.org/sessions/view/36