yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #91402
[Bug 2008716] [NEW] nova api_db sync fails remove unused build_requests columns
Public bug reported:
Description
===========
During an upgrade from xena to yoga, I followed upgrade steps using openstack-ansible to upgrade nova. During this time i was met with the following log message:
alembic.runtime.migration [-] Running upgrade d67eeaabee36 ->
b30f573d3377, Remove unused build_requests columns
And the python exception:
/openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
An error has occurred:
Traceback (most recent call last):
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 3402, in main
ret = fn(*fn_args, **fn_kwargs)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 612, in sync
return migration.db_sync(version, database='api')
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 154, in db_sync
_upgrade_alembic(engine, config, version)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 109, in _upgrade_alembic
alembic_api.upgrade(config, version or 'head')
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
script.run_env()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 848, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 117, in <module>
run_migrations_online()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 111, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
step.migration_fn(**kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/versions/b30f573d3377_remove_unused_build_requests_columns.py", line 45, in upgrade
batch_op.drop_column('request_spec_id')
File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
impl.flush()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/batch.py", line 108, in flush
fn(*arg, **kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 329, in drop_column
self._exec(base.DropColumn(table_name, column, schema=schema))
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 193, in _exec
return conn.execute(construct, multiparams)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1289, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
return connection._execute_ddl(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_ddl
ret = self._execute_context(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
self._handle_dbapi_exception(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2024, in _handle_dbapi_exception
util.raise_(newraise, with_traceback=exc_info[2], from_=e)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
[SQL: ALTER TABLE build_requests DROP COLUMN vm_state]
Steps to reproduce
==================
I was able to reproduce this bug by dropping the entire nova database, creating a nova database and running /openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Expected result
===============
I expect nova-manage to test if build_requests is present before attempting to alter/drop it.
Actual result
=============
The python module reports the exception mentioned above.
Environment
===============
Nova: 25.0.2
Openstack: Yoga
Mariadb: 10.6.11
Hypervisor: libvert+kvm
Storage Type: ceph
Network: Neutron
** Affects: nova
Importance: Undecided
Status: New
** Tags: db
** Tags added: db
** Description changed:
Description
===========
During an upgrade from wallaby to yoga, I followed upgrade steps using openstack-ansible to upgrade nova. During this time i was met with the following log message:
alembic.runtime.migration [-] Running upgrade d67eeaabee36 ->
b30f573d3377, Remove unused build_requests columns
And the python exception:
/openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
An error has occurred:
Traceback (most recent call last):
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
- self.dialect.do_execute(
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
- cursor.execute(statement, parameters)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
- result = self._query(query)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
- conn.query(q)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
- self._affected_rows = self._read_query_result(unbuffered=unbuffered)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
- result.read()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
- first_packet = self.connection._read_packet()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
- packet.raise_for_error()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
- err.raise_mysql_exception(self._data)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
- raise errorclass(errno, errval)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
+ self.dialect.do_execute(
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
+ cursor.execute(statement, parameters)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
+ result = self._query(query)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
+ conn.query(q)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
+ self._affected_rows = self._read_query_result(unbuffered=unbuffered)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
+ result.read()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
+ first_packet = self.connection._read_packet()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
+ packet.raise_for_error()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
+ err.raise_mysql_exception(self._data)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
+ raise errorclass(errno, errval)
pymysql.err.OperationalError: (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 3402, in main
- ret = fn(*fn_args, **fn_kwargs)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 612, in sync
- return migration.db_sync(version, database='api')
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 154, in db_sync
- _upgrade_alembic(engine, config, version)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 109, in _upgrade_alembic
- alembic_api.upgrade(config, version or 'head')
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
- script.run_env()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
- util.load_python_file(self.dir, "env.py")
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
- module = load_module_py(module_id, path)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
- spec.loader.exec_module(module) # type: ignore
- File "<frozen importlib._bootstrap_external>", line 848, in exec_module
- File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 117, in <module>
- run_migrations_online()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 111, in run_migrations_online
- context.run_migrations()
- File "<string>", line 8, in run_migrations
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
- self.get_context().run_migrations(**kw)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
- step.migration_fn(**kw)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/versions/b30f573d3377_remove_unused_build_requests_columns.py", line 45, in upgrade
- batch_op.drop_column('request_spec_id')
- File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
- next(self.gen)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
- impl.flush()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/batch.py", line 108, in flush
- fn(*arg, **kw)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 329, in drop_column
- self._exec(base.DropColumn(table_name, column, schema=schema))
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 193, in _exec
- return conn.execute(construct, multiparams)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1289, in execute
- return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
- return connection._execute_ddl(
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_ddl
- ret = self._execute_context(
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
- self._handle_dbapi_exception(
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2024, in _handle_dbapi_exception
- util.raise_(newraise, with_traceback=exc_info[2], from_=e)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
- raise exception
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
- self.dialect.do_execute(
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
- cursor.execute(statement, parameters)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
- result = self._query(query)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
- conn.query(q)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
- self._affected_rows = self._read_query_result(unbuffered=unbuffered)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
- result.read()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
- first_packet = self.connection._read_packet()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
- packet.raise_for_error()
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
- err.raise_mysql_exception(self._data)
- File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
- raise errorclass(errno, errval)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 3402, in main
+ ret = fn(*fn_args, **fn_kwargs)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 612, in sync
+ return migration.db_sync(version, database='api')
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 154, in db_sync
+ _upgrade_alembic(engine, config, version)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 109, in _upgrade_alembic
+ alembic_api.upgrade(config, version or 'head')
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
+ script.run_env()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
+ util.load_python_file(self.dir, "env.py")
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
+ module = load_module_py(module_id, path)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
+ spec.loader.exec_module(module) # type: ignore
+ File "<frozen importlib._bootstrap_external>", line 848, in exec_module
+ File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 117, in <module>
+ run_migrations_online()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 111, in run_migrations_online
+ context.run_migrations()
+ File "<string>", line 8, in run_migrations
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
+ self.get_context().run_migrations(**kw)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
+ step.migration_fn(**kw)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/versions/b30f573d3377_remove_unused_build_requests_columns.py", line 45, in upgrade
+ batch_op.drop_column('request_spec_id')
+ File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
+ next(self.gen)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
+ impl.flush()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/batch.py", line 108, in flush
+ fn(*arg, **kw)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 329, in drop_column
+ self._exec(base.DropColumn(table_name, column, schema=schema))
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 193, in _exec
+ return conn.execute(construct, multiparams)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1289, in execute
+ return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
+ return connection._execute_ddl(
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_ddl
+ ret = self._execute_context(
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
+ self._handle_dbapi_exception(
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2024, in _handle_dbapi_exception
+ util.raise_(newraise, with_traceback=exc_info[2], from_=e)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
+ raise exception
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
+ self.dialect.do_execute(
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
+ cursor.execute(statement, parameters)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
+ result = self._query(query)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
+ conn.query(q)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
+ self._affected_rows = self._read_query_result(unbuffered=unbuffered)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
+ result.read()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
+ first_packet = self.connection._read_packet()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
+ packet.raise_for_error()
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
+ err.raise_mysql_exception(self._data)
+ File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
+ raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
[SQL: ALTER TABLE build_requests DROP COLUMN vm_state]
Steps to reproduce
==================
I was able to reproduce this bug by dropping the entire nova database, creating a nova database and running /openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Expected result
===============
I expect nova-manage to test if build_requests is present before attempting to alter/drop it.
Actual result
=============
The python module reports the exception mentioned above.
Environment
===============
- Nova: 25.2.1.dev3
+ Nova: 25.0.2
Openstack: Yoga
Mariadb: 10.6.11
Hypervisor: libvert+kvm
Storage Type: ceph
Network: Neutron
** Description changed:
Description
===========
- During an upgrade from wallaby to yoga, I followed upgrade steps using openstack-ansible to upgrade nova. During this time i was met with the following log message:
+ During an upgrade from xena to yoga, I followed upgrade steps using openstack-ansible to upgrade nova. During this time i was met with the following log message:
alembic.runtime.migration [-] Running upgrade d67eeaabee36 ->
b30f573d3377, Remove unused build_requests columns
And the python exception:
/openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
An error has occurred:
Traceback (most recent call last):
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 3402, in main
ret = fn(*fn_args, **fn_kwargs)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 612, in sync
return migration.db_sync(version, database='api')
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 154, in db_sync
_upgrade_alembic(engine, config, version)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 109, in _upgrade_alembic
alembic_api.upgrade(config, version or 'head')
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
script.run_env()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 848, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 117, in <module>
run_migrations_online()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 111, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
step.migration_fn(**kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/versions/b30f573d3377_remove_unused_build_requests_columns.py", line 45, in upgrade
batch_op.drop_column('request_spec_id')
File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
impl.flush()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/batch.py", line 108, in flush
fn(*arg, **kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 329, in drop_column
self._exec(base.DropColumn(table_name, column, schema=schema))
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 193, in _exec
return conn.execute(construct, multiparams)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1289, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
return connection._execute_ddl(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_ddl
ret = self._execute_context(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
self._handle_dbapi_exception(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2024, in _handle_dbapi_exception
util.raise_(newraise, with_traceback=exc_info[2], from_=e)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
[SQL: ALTER TABLE build_requests DROP COLUMN vm_state]
Steps to reproduce
==================
I was able to reproduce this bug by dropping the entire nova database, creating a nova database and running /openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Expected result
===============
I expect nova-manage to test if build_requests is present before attempting to alter/drop it.
Actual result
=============
The python module reports the exception mentioned above.
Environment
===============
Nova: 25.0.2
Openstack: Yoga
Mariadb: 10.6.11
Hypervisor: libvert+kvm
Storage Type: ceph
Network: Neutron
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2008716
Title:
nova api_db sync fails remove unused build_requests columns
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
During an upgrade from xena to yoga, I followed upgrade steps using openstack-ansible to upgrade nova. During this time i was met with the following log message:
alembic.runtime.migration [-] Running upgrade d67eeaabee36 ->
b30f573d3377, Remove unused build_requests columns
And the python exception:
/openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code.
An error has occurred:
Traceback (most recent call last):
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 3402, in main
ret = fn(*fn_args, **fn_kwargs)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/cmd/manage.py", line 612, in sync
return migration.db_sync(version, database='api')
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 154, in db_sync
_upgrade_alembic(engine, config, version)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/migration.py", line 109, in _upgrade_alembic
alembic_api.upgrade(config, version or 'head')
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/command.py", line 320, in upgrade
script.run_env()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/script/base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 848, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 117, in <module>
run_migrations_online()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/env.py", line 111, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/runtime/migration.py", line 620, in run_migrations
step.migration_fn(**kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/nova/db/api/migrations/versions/b30f573d3377_remove_unused_build_requests_columns.py", line 45, in upgrade
batch_op.drop_column('request_spec_id')
File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
next(self.gen)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/base.py", line 374, in batch_alter_table
impl.flush()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/operations/batch.py", line 108, in flush
fn(*arg, **kw)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 329, in drop_column
self._exec(base.DropColumn(table_name, column, schema=schema))
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/alembic/ddl/impl.py", line 193, in _exec
return conn.execute(construct, multiparams)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1289, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
return connection._execute_ddl(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1381, in _execute_ddl
ret = self._execute_context(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
self._handle_dbapi_exception(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2024, in _handle_dbapi_exception
util.raise_(newraise, with_traceback=exc_info[2], from_=e)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
self.dialect.do_execute(
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute
cursor.execute(statement, parameters)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/openstack/venvs/nova-25.2.1.dev3/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1091, "Can't DROP COLUMN `vm_state`; check that it exists")
[SQL: ALTER TABLE build_requests DROP COLUMN vm_state]
Steps to reproduce
==================
I was able to reproduce this bug by dropping the entire nova database, creating a nova database and running /openstack/venvs/nova-25.2.1.dev3/bin/nova-manage api_db sync
Expected result
===============
I expect nova-manage to test if build_requests is present before attempting to alter/drop it.
Actual result
=============
The python module reports the exception mentioned above.
Environment
===============
Nova: 25.0.2
Openstack: Yoga
Mariadb: 10.6.11
Hypervisor: libvert+kvm
Storage Type: ceph
Network: Neutron
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2008716/+subscriptions
Follow ups
