openstack team mailing list archive

[Jim Fehlig <jfehlig@xxxxxxxx>]

Adam Young wrote:
> On 07/16/2012 11:59 PM, Jim Fehlig wrote:
>> I'm working on a patch that adds a column to the compute_nodes table in
>> the nova db, but it seems my db migration script fails when calling 'db
>> sync' in stack.sh.  I tried running the command manually, same failure:
>>
>> stack@virt71:~> /opt/stack/nova/bin/nova-manage --debug -v db
>> sync2012-07-16 21:42:52 DEBUG nova.utils [-] backend <module
>> 'nova.db.sqlalchemy.migration' from
>> '/opt/stack/nova/nova/db/sqlalchemy/migration.pyc'> from (pid=19230)
>> __get_backend /opt/stack/nova/nova/utils.py:484
>> /usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:681:
>> SADeprecationWarning: The 'listeners' argument to Pool (and
>> create_engine()) is deprecated.  Use event.listen().
>>    Pool.__init__(self, creator, **kw)
>> /usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:159:
>> SADeprecationWarning: Pool.add_listener is deprecated.  Use
>> event.listen()
>>    self.add_listener(l)
>> Command failed, please check log for more info
>>
>> I can't find anything useful in any log (/var/log/*, /opt/stack/log/*).
>> I ran the above in strace and saw my migration script was opened and
>> then shortly thereafter writing of "Command failed, please check log for
>> more info" and exit(1) :).
>>
>> The patch also adds a 'hypervisor_type' column to the instances table,
>> and that migration script succeeds!
>>
>> Any hints for debugging a db migration script?
>>
>> Thanks,
>> Jim
>>
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>
> I just went through this with a Keystone change.  What I would suggest
> is:
>
> 1.  Get a blank Database.
> 2.  Run the DB migration without  your scripts.
> 3. Get the SQL you want to run to work correctly from that step.
> 4. Add in a Database appropriate SQL script that has exactly your SQL
> from above.
> 5. Run the whole migration.
>
> Yes, it is a labor intensive and painful as it sounds.  You want to
> make sure that you have exactly the preconditions that your script
> expects.  What I had to do was actually go back and modify earlier DB
> init code due to the SQL alchemy column definition changing.
>
> Note this change:
> https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/001_add_initial_tables.py
>
> I now have to explicitly create the token table to make sure it is the
> state it would be today.  Since my code had modified the token table,
> had I not done this,  by the end of "stage 1" SQL processing, the
> database would have had this table in "stage 2" state.
>
> Then I Went and added a SQL script for modifying the table.  Since I
> was altering a a table without dumping the data in it, it was a
> non-trivial change that SQL Alchemy couldn't handl (AFAICT). Instead,
> I added a SQL script:
> https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/002_mysql_upgrade.sql
>
>
> Make sure you have a comparable Downgrade script, too.
>
> https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/002_mysql_downgrade.sql

Thanks for the details Adam!  This is really helpful.

Regards,
Jim

>
>
>
> For Keystone, we run the upgrade using a stand alone executable
> keystone/bin/keystone-manage.  In nove, it looks like there is
> bin/nova-manage to do the same thing.  I am running using Eclipse and
> PyDev as my development environment, and using the integrated debugger
> to step through the code.  Makes it a lot less painful to debug.
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>