openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On Mon, Mar 21, 2011 at 1:08 PM, Justin Santa Barbara
<justin@xxxxxxxxxxxx> wrote:
>> By "seemed powerful", did you actually try it on anything more than
>> simple add a table/drop a table scripts? Did you try it on more than a
>> single database? Did you use the SQL scripting as well as the Python
>> change scripts?
>
> I've written 3 or 4 - most did the simple stuff, but one also added a column
> with a non-null default value.  I stayed away from SQL scripting because my
> understanding is that the whole idea of SA is that we're not using raw SQL.
> My migrations should have been easy, but SA-migrate did made them harder.
>  However, it was the documentation that really made it much harder; for

The docs only cover the most basic things, yeah. Unfortunately, the
test cases in sa-migrate *also* only cover the most basic of
operations, and many migrate patterns that are common, but more
complex than "add a table or a column", are entirely untested code
paths. And those code paths have bugs, as I mentioned in the original
post. :(

> example, it seems you should use reflection on a table you're modifying, but
> that was mostly absent from the docs.

Sure, you can use reflection, but actually, table reflection is what
sa-migrate is doing "under the covers" when it processes a Python
change script. Unfortunately, it's that reflection that produces the
bug #1. It's a long explanation, but you can look in the code to
verify that statement....

> I found some posts where the mailing
> list guys stated what I had to do for my 'non-null default' (create a
> NULLable column, set the default value, alter the column it is non-NULL),
> but there wasn't an actual code example, which would have made it trivial.

Agreed, as mentioned, the examples are all (too) simplistic.

>  However, now that it's done, it would be easy to do again.  My experience
> suggests that this is a brick-wall learning curve that we have to get over.

It's not a learning curve. It's bugs in sa-migrate that are the main
issue for Glance. As mentioned, bug #1's solution is to use SQL
scripts, but multi-statemetn SQL scripts don't work properly for
MySQL. So you have an untenable solution.

> Jay and I talked on IRC, and he's doing something much more complicated than
> my "supposed to be simple" migration (he's moving a column between tables,
> and testing with downgrades also), and his experience is that the brick wall
> is un-scalable (and that the grass on the other side isn't even that green.)
>  So Jay: if you want to use SQL scripts to unblock glance, then go for it.
>  It puts the SA-migrate code in an odd limbo, which is why I'd prefer that
> we were able to have SA-migrate work, but if the developer "on the ground"
> says it doesn't work, then we should go by that.
> I guess we should probably discuss the broader implications at the Design
> Summit?

Not sure. I posted this to get the discussion going and was hoping
someone might have suggestions showing some alternative solution that
I hadn't thought of.

Cheers!
jay