openstack team mailing list archive

[Adam Young <ayoung@xxxxxxxxxx>]

On 07/16/2012 11:59 PM, Jim Fehlig wrote:
> I'm working on a patch that adds a column to the compute_nodes table in
> the nova db, but it seems my db migration script fails when calling 'db
> sync' in stack.sh.  I tried running the command manually, same failure:
>
> stack@virt71:~> /opt/stack/nova/bin/nova-manage --debug -v db
> sync2012-07-16 21:42:52 DEBUG nova.utils [-] backend <module
> 'nova.db.sqlalchemy.migration' from
> '/opt/stack/nova/nova/db/sqlalchemy/migration.pyc'> from (pid=19230)
> __get_backend /opt/stack/nova/nova/utils.py:484
> /usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:681:
> SADeprecationWarning: The 'listeners' argument to Pool (and
> create_engine()) is deprecated.  Use event.listen().
>    Pool.__init__(self, creator, **kw)
> /usr/lib64/python2.6/site-packages/sqlalchemy/pool.py:159:
> SADeprecationWarning: Pool.add_listener is deprecated.  Use event.listen()
>    self.add_listener(l)
> Command failed, please check log for more info
>
> I can't find anything useful in any log (/var/log/*, /opt/stack/log/*).
> I ran the above in strace and saw my migration script was opened and
> then shortly thereafter writing of "Command failed, please check log for
> more info" and exit(1) :).
>
> The patch also adds a 'hypervisor_type' column to the instances table,
> and that migration script succeeds!
>
> Any hints for debugging a db migration script?
>
> Thanks,
> Jim
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

I just went through this with a Keystone change.  What I would suggest is:

1.  Get a blank Database.
2.  Run the DB migration without  your scripts.
3. Get the SQL you want to run to work correctly from that step.
4. Add in a Database appropriate SQL script that has exactly your SQL 
from above.
5. Run the whole migration.

Yes, it is a labor intensive and painful as it sounds.  You want to make 
sure that you have exactly the preconditions that your script expects.  
What I had to do was actually go back and modify earlier DB init code 
due to the SQL alchemy column definition changing.

Note this change:
https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/001_add_initial_tables.py
I now have to explicitly create the token table to make sure it is the 
state it would be today.  Since my code had modified the token table, 
had I not done this,  by the end of "stage 1" SQL processing, the 
database would have had this table in "stage 2" state.

Then I Went and added a SQL script for modifying the table.  Since I was 
altering a a table without dumping the data in it, it was a non-trivial 
change that SQL Alchemy couldn't handl (AFAICT). Instead, I added a SQL 
script:
https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/002_mysql_upgrade.sql

Make sure you have a comparable Downgrade script, too.

https://review.openstack.org/#/c/7754/9/keystone/common/sql/migrate_repo/versions/002_mysql_downgrade.sql


For Keystone, we run the upgrade using a stand alone executable 
keystone/bin/keystone-manage.  In nove, it looks like there is 
bin/nova-manage to do the same thing.  I am running using Eclipse and 
PyDev as my development environment, and using the integrated debugger 
to step through the code.  Makes it a lot less painful to debug.