widelands-dev team mailing list archive

[Hans Joachim Desserud <launchpad@xxxxxxxxxxxx>]

>I agree - but right now we do not seem to have clear steps forward for this.

First of all, I feel I should apologize for the delay. The messages back and forth on this merge proposal is quite sporadic. I do care about this issue, but for some reason (though partially because I don't know Django all that well) I find that I never really get round to working on this. 

I can say abit about what I found out last time I looked into this. I investigated south for database migrations which I was happy (and surprised) to find that we already use. Seems to at least have been used for pybb/migrations, though I don't know the details nor who used this.

The good news is that South is able to autogenerate migrations based on what it knows about the existing database state when the models.py for a module changes. Though this require that it actually knows about the initial state in order to tell what has changed.

So my battle plan/sketch is as follows:
1. We need to map out the modules and related database tables which will need to be migrated during the upgrade so that we can record the intial/current state for these. (Alternatively record the current state for all tables so that we can more easily work with database migrations in the future)
2. For these tables, South can automatically generate a migration which would create/setup the tables based on how they are defined in models.py. These would be refered to as intial migrations. (I would link to the South documentation on how to do this, but it has been a while since I read about it.)
3. The migrations from 2 should be gathered in a separate merge proposal and pushed to production. There's a command which can be run to perform the migrations. Here's the CRITICAL step: These initial migrations should be faked using --fake [1]. This doesn't actually run the migrations, but stores them in the migrations table to record that they have been run. The reason for doing this is because we are letting the migration system know what exists, if actually run the intial migrations it will assume no such tables exist and attempt to create them (from what I read, possibly clear the data if they do exist!)
4. At this point we are still running the old code, but we are now aware of the current database state. Now we return to this merge proposal and continue working on it. Check out this code on a local machine, review and pull in the updated dependencies for it.
5. With the new version of dependencies and their modules in place, I believe it should be possible to simply run South to generate the necessary migrations to get from the known state in the database to the modules.py defined in these newer dependencies.
6. This should then be thoroughly tested against a copy of the prod database to ensure things still remain working.
7. When rolling out in prod (or for the test in the previous code), it would consist of the following steps:
1) Stop the app.
2) Pull in the latest code changes
3) Run the migrations to get the DB up to date
4) Start the app again.
5) Verify things are working
6) Be happy :)

So that's a plan at least. As you notice it would be split across two independent deployments. The intial migrations could possibly be loaded at any time, but they would need to be in place before we attempt the upgrade. Note that all of the above is completely untested, but from the impression I got from the documentation and talking abit with people on IRC, it should be doable.

I think the next step would be to start working on the intial migrations, and run quickly through this to check if my theory holds water. 

[1] http://south.readthedocs.org/en/latest/commands.html#options
-- 
https://code.launchpad.net/~shevonar/widelands-website/django1.4/+merge/143219
Your team Widelands Developers is requested to review the proposed merge of lp:~shevonar/widelands-website/django1.4 into lp:widelands-website.