openstack team mailing list archive

[Dolph Mathews <dolph.mathews@xxxxxxxxx>]

Looking for some support from anyone with experience with
sqlalchemy-migrate on the following review...

https://review.openstack.org/#change,1200

See my Nov 1 comment -- sqlalchemy's built-in `./manage.py test` command
fails, but you can test each migration individually (forward & backward)
and they appear to work fine. It appears as though the test command is
attempting to apply the third/last schema revision without running the
prior migrations first? I'm hoping someone can tell me I'm doing something
dumb...

Thanks!

-Dolph

On Fri, Oct 28, 2011 at 12:55 PM, Ziad Sawalha
<ziad.sawalha@xxxxxxxxxxxxx>wrote:

>  Thanks, pvo. This looks like good. We'd like to follow what other
> projects do. Has there been any feedback on this BP? Will this be "the way"
> to handle things in OpenStack?
>
>  The easiest way for us to handle this now for schema migrations to
> Keystone is to follow Jesse's suggestion and use SQL Alchemy migrate_repo.
> We'll do that for schema changes currently in our branches and will work
> towards adopting the BP above.
>
>  Z
>
>
>
>   From: Paul Voccio <paul.voccio@xxxxxxxxxxxxx>
> Date: Tue, 25 Oct 2011 19:33:57 -0500
> To: Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx>, Ziad Sawalha <
> ziad.sawalha@xxxxxxxxxxxxx>
> Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
> Subject: Re: [Openstack] Handling Schema Changes in Keystone
>
>   There is a BP out for how to handle continuious delivery and db
> upgrades within Nova. It does address some of the issues that keystone may
> face but feedback on the doc would be appreciated.
>
>  https://blueprints.launchpad.net/nova/+spec/deployability-improvements
>
>  pvo
>
>   From: Brian Schott <brian.schott@xxxxxxxxxxxxxxxxxx>
> Date: Tue, 25 Oct 2011 18:16:30 -0400
> To: Ziad Sawalha <ziad.sawalha@xxxxxxxxxxxxx>
> Cc: "openstack@xxxxxxxxxxxxxxxxxxx" <openstack@xxxxxxxxxxxxxxxxxxx>
> Subject: Re: [Openstack] Handling Schema Changes in Keystone
>
>   It's probably the best approach that doesn't depend on the Django ORM
> (like South).  Does anyone use the "experimental" command line migrate to
> generate the migrate script?  I always did it by hand.
> http://packages.python.org/sqlalchemy-migrate/
>
>  On the dev side, one of the big headaches in nova migrate_repo has been
> that the file numbering by hand meant that competing feature teams that
> needed to incorporate a schema change had to keep bumping the number every
> time a new version file got committed.  We talked about relaxing
> migrate_repo numbering rules so that you could create a
> 999_my_pending_change.py file that didn't break the version numbering
> adjacency checks.  I don't know if that happened.  Otherwise, every time a
> new file arrived in trunk, we all had to manually renumber our development
> files from 055_x to 056_x...  Not a big deal, but annoying when you pushed
> a branch up for review, then it broke because something else arrived in the
> same number slot.
>
>  On the deploy side, complicated table transforms don't always map well
> to all database backends and I don't think here is any unit testing with
> populated fixtures for data upgrade/downgrade. Don't know if this has been
> an issue in real deployments, but the opportunity for sysadmin excellence
> is certainly there....
>
>  Brian
>
>     -------------------------------------------------
> Brian Schott, CTO
> Nimbis Services, Inc.
> brian.schott@xxxxxxxxxxxxxxxxxx
> ph: 443-274-6064  fx: 443-274-6060
>
>
>
>
>
>
>
>  On Oct 25, 2011, at 5:45 PM, Ziad Sawalha wrote:
>
>   Our schema right now is auto generated from the model using sqlalchemy.
> Whenever we change the model, the generated schema is different for new
> installations but this does not address existing deployments.
>
>  Looking for feedback on how to handle this better:
>  anotherjesses offered:
> https://github.com/openstack/nova/tree/master/nova/db/sqlalchemy/migrate_repo
>
>  Questions:
> - Has anyone used this on the dev side?
> - Has anyone used this on the deployment/ops side?
>
>  Would love to hear from you how you started it (we have multiple
> versions of our schema out there, so where do we start) and what was the
> experience updating versions.
>
>  Regards,
>
>  Ziad
>  _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>
>    _______________________________________________ Mailing list:
> https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netUnsubscribe :
> https://launchpad.net/~openstack More help :
> https://help.launchpad.net/ListHelp
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>
>