← Back to team overview

maas-devel team mailing list archive

Re: Database Migrations and backports

 

* We must be very careful when adding new migrations that nobody
else is working on one too, because one of you must base your
migration off the other's

The reason why this situation is bad (but not catastrophic) is that when we do this, South will use the state of the whole db, as recorded at the end of the last migration file to compute the changes to models. If two migration number 10 where added (10a and 10b), one of them will be 'the last one' (say, 10a). 10a won't have the record of what happened in 10b and thus, next time one creates a migration, the changes in 10b will be seen by South as recent changes.

The solution is to manually create an empty migration (./bin/maas schemamigration --empty maasserver merge_models) after having merged the two branches. This new migration will contain an accurate description of the state of the models.

* When amending old release branches (e.g. lp:maas/1.2) we *cannot*
add new migrations to it on their own, we *must* add it to trunk
and backport all the missing trunk migrations.  If this is not
done, it will break upgrades between releases.

That situation would be much more problematic as it would block us from moving forward in the future. I think we can reasonably say that any DB migration done on the 1.2 branch must be done on trunk as well.

1) Create 0043 in branch 1.2. Merge that into trunk which also had an
    0043 patch and an 0044 patch.
    Upgrade worked correctly from 1.2 => trunk.

This is the first situation I described. The only problem might be when someone else tries to create another migration. (Because the changes from your migration 0043 are not recorded in the state of the db 0044.)

2) Trying to simply rename 0043 => 0045 in trunk fails because the DB
    doesn't think it has the patch and tries to apply it twice

Internally, South keeps a record of the name of the applied migrations (the table is named 'south_migrationhistory') so renaming a migration is not an option.

3) Creating the patch as 0045 in trunk, and then putting just that into
    1.2. It successfully migrates 1.2, adding the 0045 patch. However it
    fails to apply 0043 and 0044 when upgrading to trunk because they
    appear out of order. "migrate --merge" works, but isn't our default
    setting.

4) Obviously creating it on trunk and merging all db migrations works.


So while what you are recommending (4) works, I'd like to point out
that it means we can never land anything on trunk that you don't want
to backport to 1.2 if we ever want to land anything else from trunk on
1.2.

I'd like to point out that (1) seemed to work quite well in my
testing. So we might consider that instead of creating patches on
trunk, and having to backport all pending DB migrations back to 1.2,
we might want to try creating DB migrations on 1.2 and merging up into
trunk where we need it.

Actually, (1) is exactly the same situation as (3), it just breaks for different use-cases. Imagine someone using a package with the migrations 0043 and 0044. When the fix containing your new migration 0043 will be available and packaged, it will simply break the upgrade.

I think we want to be able to land DB migrations in trunk without having to land them in 1.2. For that to work, the only way is to: - make sure that *all* the DB migrations are landed on trunk. (Some of them will be missing in 1.2) - change the packaging to use '--merge' when applying the migrations. The only problem with that would a semantic problem between migrations but that should be picked up during the testing phase.

R.


Follow ups

References