← Back to team overview

openstack team mailing list archive

Re: debugging a db migration script

 

On 07/16/2012 11:59 PM, Jim Fehlig wrote:
I'm working on a patch that adds a column to the compute_nodes table in
the nova db, but it seems my db migration script fails when calling 'db
sync' in stack.sh.  I tried running the command manually, same failure:

stack@virt71:~> /opt/stack/nova/bin/nova-manage --debug -v db
sync2012-07-16 21:42:52 DEBUG nova.utils [-] backend <module
'nova.db.sqlalchemy.migration' from
'/opt/stack/nova/nova/db/sqlalchemy/migration.pyc'> from (pid=19230)
__get_backend /opt/stack/nova/nova/utils.py:484
/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:681:
SADeprecationWarning: The 'listeners' argument to Pool (and
create_engine()) is deprecated.  Use event.listen().
   Pool.__init__(self, creator, **kw)
/usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:159:
SADeprecationWarning: Pool.add_listener is deprecated.  Use event.listen()
   self.add_listener(l)
Command failed, please check log for more info

I can't find anything useful in any log (/var/log/*, /opt/stack/log/*).
I ran the above in strace and saw my migration script was opened and
then shortly thereafter writing of "Command failed, please check log for
more info" and exit(1) :).

The patch also adds a 'hypervisor_type' column to the instances table,
and that migration script succeeds!

Any hints for debugging a db migration script?

Thanks,
Jim


_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

I just went through this with a Keystone change.  What I would suggest is:

1.  Get a blank Database.
2.  Run the DB migration without  your scripts.
3. Get the SQL you want to run to work correctly from that step.
4. Add in a Database appropriate SQL script that has exactly your SQL from above.
5. Run the whole migration.

Yes, it is a labor intensive and painful as it sounds. You want to make sure that you have exactly the preconditions that your script expects. What I had to do was actually go back and modify earlier DB init code due to the SQL alchemy column definition changing.

Note this change:
https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/001_add_initial_tables.py
I now have to explicitly create the token table to make sure it is the state it would be today. Since my code had modified the token table, had I not done this, by the end of "stage 1" SQL processing, the database would have had this table in "stage 2" state.

Then I Went and added a SQL script for modifying the table. Since I was altering a a table without dumping the data in it, it was a non-trivial change that SQL Alchemy couldn't handl (AFAICT). Instead, I added a SQL script:
https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/002_mysql_upgrade.sql

Make sure you have a comparable Downgrade script, too.

https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/002_mysql_downgrade.sql


For Keystone, we run the upgrade using a stand alone executable keystone/bin/keystone-manage. In nove, it looks like there is bin/nova-manage to do the same thing. I am running using Eclipse and PyDev as my development environment, and using the integrated debugger to step through the code. Makes it a lot less painful to debug.




Follow ups

References