openstack team mailing list archive

[Dolph Mathews <dolph.mathews@xxxxxxxxx>]

Yogi just answered my question below -- In short, the ./manage.py test
command didn't behave as I understood.

It simply tests the latest migration forwards & backwards, not the entire
history of migrations in sequence.

Thanks, Yogi!

-Dolph

On Fri, Nov 4, 2011 at 10:37 AM, Dolph Mathews <dolph.mathews@xxxxxxxxx>wrote:

> Looking for some support from anyone with experience with
> sqlalchemy-migrate on the following review...
>
> https://review.openstack.org/#change,1200
>
> See my Nov 1 comment -- sqlalchemy's built-in `./manage.py test` command
> fails, but you can test each migration individually (forward & backward)
> and they appear to work fine. It appears as though the test command is
> attempting to apply the third/last schema revision without running the
> prior migrations first? I'm hoping someone can tell me I'm doing something
> dumb...
>
> Thanks!
>
> -Dolph
>
>
> On Fri, Oct 28, 2011 at 12:55 PM, Ziad Sawalha <ziad.sawalha@xxxxxxxxxxxxx
> > wrote:
>
>>  Thanks, pvo. This looks like good. We'd like to follow what other
>> projects do. Has there been any feedback on this BP? Will this be "the way"
>> to handle things in OpenStack?
>>
>>  The easiest way for us to handle this now for schema migrations to
>> Keystone is to follow Jesse's suggestion and use SQL Alchemy migrate_repo.
>> We'll do that for schema changes currently in our branches and will work
>> towards adopting the BP above.
>>
>>  Z
>>
>>
>>
>>   From: Paul Voccio <paul.voccio@xxxxxxxxxxxxx>
>> Date: Tue, 25 Oct 2011 19:33:57 -0500
>> To: Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx>, Ziad Sawalha <
>> ziad.sawalha@xxxxxxxxxxxxx>
>> Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
>> Subject: Re: [Openstack] Handling Schema Changes in Keystone
>>
>>   There is a BP out for how to handle continuious delivery and db
>> upgrades within Nova. It does address some of the issues that keystone may
>> face but feedback on the doc would be appreciated.
>>
>>  https://blueprints.launchpad.net/nova/+spec/deployability-improvements
>>
>>  pvo
>>
>>   From: Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx>
>> Date: Tue, 25 Oct 2011 18:16:30 -0400
>> To: Ziad Sawalha <ziad.sawalha@xxxxxxxxxxxxx>
>> Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
>> Subject: Re: [Openstack] Handling Schema Changes in Keystone
>>
>>   It's probably the best approach that doesn't depend on the Django ORM
>> (like South).  Does anyone use the "experimental" command line migrate to
>> generate the migrate script?  I always did it by hand.
>> http://packages.python.org/sqlalchemy-migrate/
>>
>>  On the dev side, one of the big headaches in nova migrate_repo has been
>> that the file numbering by hand meant that competing feature teams that
>> needed to incorporate a schema change had to keep bumping the number every
>> time a new version file got committed.  We talked about relaxing
>> migrate_repo numbering rules so that you could create a
>> 999_my_pending_change.py file that didn't break the version numbering
>> adjacency checks.  I don't know if that happened.  Otherwise, every time a
>> new file arrived in trunk, we all had to manually renumber our development
>> files from 055_x to 056_x...  Not a big deal, but annoying when you pushed
>> a branch up for review, then it broke because something else arrived in the
>> same number slot.
>>
>>  On the deploy side, complicated table transforms don't always map well
>> to all database backends and I don't think here is any unit testing with
>> populated fixtures for data upgrade/downgrade. Don't know if this has been
>> an issue in real deployments, but the opportunity for sysadmin excellence
>> is certainly there....
>>
>>  Brian
>>
>>     -------------------------------------------------
>> Brian Schott, CTO
>> Nimbis Services, Inc.
>> brian.schott@xxxxxxxxxxxxxxxxxx
>> ph: 443-274-6064  fx: 443-274-6060
>>
>>
>>
>>
>>
>>
>>
>>  On Oct 25, 2011, at 5:45 PM, Ziad Sawalha wrote:
>>
>>   Our schema right now is auto generated from the model using
>> sqlalchemy. Whenever we change the model, the generated schema is different
>> for new installations but this does not address existing deployments.
>>
>>  Looking for feedback on how to handle this better:
>>  anotherjesses offered:
>> https://github.com/openstack/nova/tree/master/nova/db/sqlalchemy/migrate_repo
>>
>>  Questions:
>> - Has anyone used this on the dev side?
>> - Has anyone used this on the deployment/ops side?
>>
>>  Would love to hear from you how you started it (we have multiple
>> versions of our schema out there, so where do we start) and what was the
>> experience updating versions.
>>
>>  Regards,
>>
>>  Ziad
>>  _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>>    _______________________________________________ Mailing list:
>> https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netUnsubscribe :
>> https://launchpad.net/~openstack More help :
>> https://help.launchpad.net/ListHelp
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>