yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #93975
[Bug 2065693] [NEW] delete server fails with Cannot add or update a child row: a foreign key constraint fails
Public bug reported:
Test fails as:-
Traceback (most recent call last):
File "/opt/stack/tempest/tempest/api/compute/servers/test_delete_server.py", line 44, in test_delete_server_while_in_building_state
self.client.delete_server(server['id'])
File "/opt/stack/tempest/tempest/lib/services/compute/servers_client.py", line 170, in delete_server
resp, body = self.delete("servers/%s" % server_id)
File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 339, in delete
return self.request('DELETE', url, extra_headers, headers, body)
File "/opt/stack/tempest/tempest/lib/services/compute/base_compute_client.py", line 47, in request
resp, resp_body = super(BaseComputeClient, self).request(
File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 742, in request
self._error_checker(resp, resp_body)
File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 922, in _error_checker
raise exceptions.ServerFault(resp_body, resp=resp,
tempest.lib.exceptions.ServerFault: Got server fault
Details: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_db.exception.DBReferenceError'>
Nova traces:-
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] Unexpected exception in API method: oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))')
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, action, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)]
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: [parameters: {'created_at': datetime.datetime(2024, 5, 3, 20, 10, 51, 82097), 'updated_at': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'deleted_at': None, 'deleted': 0, 'action': 'delete', 'instance_uuid': '61f3c713-8ef1-4281-a3db-f199197efda3', 'request_id': 'req-19b3c8fc-586e-40ad-b3b7-3cc604632d69', 'user_id': '2aa1943eb18440ecb0ee37b064938127', 'project_id': '6d39b8d66e43454697d88a86755b45e3', 'start_time': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'finish_time': None, 'message': None}]
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: (Background on this error at: https://sqlalche.me/e/14/gkpj)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self.dialect.do_execute(
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = self._query(query)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi conn.query(q)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result.read()
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet()
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi packet.raise_for_error()
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))')
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception:
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 665, in wrapped
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1102, in delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 905, in _delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self.compute_api.delete(context, instance)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 242, in inner
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 168, in inner
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2726, in delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._delete_instance(context, instance)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2717, in _delete_instance
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._delete(context, instance, 'delete', self._do_delete,
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2554, in _delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._record_action_start(context, instance,
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 414, in _record_action_start
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi objects.InstanceAction.action_start(context, instance.uuid,
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_action.py", line 86, in action_start
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi db_action = db.action_start(context, values)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 207, in wrapper
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 3900, in action_start
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi action_ref.save(context.session)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/models.py", line 38, in save
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi session.flush()
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3449, in flush
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._flush(objects)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3588, in _flush
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi with util.safe_reraise():
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi compat.raise_(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise exception
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3549, in _flush
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi flush_context.execute()
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi rec.execute(self)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi _emit_insert_statements(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = connection._execute_20(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi ret = self._execute_context(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2132, in _handle_dbapi_exception
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise exception
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self.dialect.do_execute(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = self._query(query)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi conn.query(q)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result.read()
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet()
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi packet.raise_for_error()
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data)
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval)
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))')
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, action, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)]
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2024, 5, 3, 20, 10, 51, 82097), 'updated_at': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'deleted_at': None, 'deleted': 0, 'action': 'delete', 'instance_uuid': '61f3c713-8ef1-4281-a3db-f199197efda3', 'request_id': 'req-19b3c8fc-586e-40ad-b3b7-3cc604632d69', 'user_id': '2aa1943eb18440ecb0ee37b064938127', 'project_id': '6d39b8d66e43454697d88a86755b45e3', 'start_time': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'finish_time': None, 'message': None}]
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj)
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: INFO nova.api.openstack.wsgi [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: <class 'oslo_db.exception.DBReferenceError'>
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: DEBUG nova.api.openstack.wsgi [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: <class 'oslo_db.exception.DBReferenceError'> {{(pid=56087) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:943}}
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: INFO nova.api.openstack.requestlog [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] 10.209.37.96 "DELETE /compute/v2.1/servers/61f3c713-8ef1-4281-a3db-f199197efda3" status: 500 len: 208 microversion: 2.1 time: 1.209063
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: [pid: 56087|app: 0|req: 914/1850] 10.209.37.96 () {66 vars in 1379 bytes} [Fri May 3 20:10:49 2024] DELETE /compute/v2.1/servers/61f3c713-8ef1-4281-a3db-f199197efda3 => generated 208 bytes in 1211 msecs (HTTP/1.1 500) 9 headers in 391 bytes (1 switches on core 0)
Seen once in:-
https://b04549659f81127bccd5-1b102124c6fd0b6d71635117190cffc0.ssl.cf1.rackcdn.com/917262/2/check/neutron-ovs-tempest-multinode-full/a21ea6a/testr_results.html
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2065693
Title:
delete server fails with Cannot add or update a child row: a foreign
key constraint fails
Status in OpenStack Compute (nova):
New
Bug description:
Test fails as:-
Traceback (most recent call last):
File "/opt/stack/tempest/tempest/api/compute/servers/test_delete_server.py", line 44, in test_delete_server_while_in_building_state
self.client.delete_server(server['id'])
File "/opt/stack/tempest/tempest/lib/services/compute/servers_client.py", line 170, in delete_server
resp, body = self.delete("servers/%s" % server_id)
File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 339, in delete
return self.request('DELETE', url, extra_headers, headers, body)
File "/opt/stack/tempest/tempest/lib/services/compute/base_compute_client.py", line 47, in request
resp, resp_body = super(BaseComputeClient, self).request(
File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 742, in request
self._error_checker(resp, resp_body)
File "/opt/stack/tempest/tempest/lib/common/rest_client.py", line 922, in _error_checker
raise exceptions.ServerFault(resp_body, resp=resp,
tempest.lib.exceptions.ServerFault: Got server fault
Details: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_db.exception.DBReferenceError'>
Nova traces:-
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] Unexpected exception in API method: oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))')
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, action, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)]
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: [parameters: {'created_at': datetime.datetime(2024, 5, 3, 20, 10, 51, 82097), 'updated_at': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'deleted_at': None, 'deleted': 0, 'action': 'delete', 'instance_uuid': '61f3c713-8ef1-4281-a3db-f199197efda3', 'request_id': 'req-19b3c8fc-586e-40ad-b3b7-3cc604632d69', 'user_id': '2aa1943eb18440ecb0ee37b064938127', 'project_id': '6d39b8d66e43454697d88a86755b45e3', 'start_time': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'finish_time': None, 'message': None}]
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: (Background on this error at: https://sqlalche.me/e/14/gkpj)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self.dialect.do_execute(
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = self._query(query)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi conn.query(q)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result.read()
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet()
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi packet.raise_for_error()
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))')
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception:
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi Traceback (most recent call last):
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 665, in wrapped
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1102, in delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 905, in _delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self.compute_api.delete(context, instance)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 242, in inner
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 168, in inner
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2726, in delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._delete_instance(context, instance)
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2717, in _delete_instance
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._delete(context, instance, 'delete', self._do_delete,
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2554, in _delete
May 03 20:10:51.134858 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._record_action_start(context, instance,
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 414, in _record_action_start
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi objects.InstanceAction.action_start(context, instance.uuid,
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_action.py", line 86, in action_start
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi db_action = db.action_start(context, values)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 207, in wrapper
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 3900, in action_start
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi action_ref.save(context.session)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/models.py", line 38, in save
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi session.flush()
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3449, in flush
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._flush(objects)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3588, in _flush
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi with util.safe_reraise():
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi compat.raise_(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise exception
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/session.py", line 3549, in _flush
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi flush_context.execute()
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi rec.execute(self)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi _emit_insert_statements(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = connection._execute_20(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi ret = self._execute_context(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2132, in _handle_dbapi_exception
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise exception
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self.dialect.do_execute(
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 153, in execute
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result = self._query(query)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/cursors.py", line 322, in _query
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi conn.query(q)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 558, in query
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered)
May 03 20:10:51.139041 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 822, in _read_query_result
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi result.read()
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 1200, in read
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet()
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/connections.py", line 772, in _read_packet
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi packet.raise_for_error()
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/protocol.py", line 221, in raise_for_error
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data)
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi File "/opt/stack/data/venv/lib/python3.10/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval)
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBReferenceError: (pymysql.err.IntegrityError) (1452, 'Cannot add or update a child row: a foreign key constraint fails (`nova_cell0`.`instance_actions`, CONSTRAINT `fk_instance_actions_instance_uuid` FOREIGN KEY (`instance_uuid`) REFERENCES `instances` (`uuid`))')
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO instance_actions (created_at, updated_at, deleted_at, deleted, action, instance_uuid, request_id, user_id, project_id, start_time, finish_time, message) VALUES (%(created_at)s, %(updated_at)s, %(deleted_at)s, %(deleted)s, %(action)s, %(instance_uuid)s, %(request_id)s, %(user_id)s, %(project_id)s, %(start_time)s, %(finish_time)s, %(message)s)]
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2024, 5, 3, 20, 10, 51, 82097), 'updated_at': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'deleted_at': None, 'deleted': 0, 'action': 'delete', 'instance_uuid': '61f3c713-8ef1-4281-a3db-f199197efda3', 'request_id': 'req-19b3c8fc-586e-40ad-b3b7-3cc604632d69', 'user_id': '2aa1943eb18440ecb0ee37b064938127', 'project_id': '6d39b8d66e43454697d88a86755b45e3', 'start_time': datetime.datetime(2024, 5, 3, 20, 10, 49, 930003), 'finish_time': None, 'message': None}]
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj)
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: ERROR nova.api.openstack.wsgi
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: INFO nova.api.openstack.wsgi [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: <class 'oslo_db.exception.DBReferenceError'>
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: DEBUG nova.api.openstack.wsgi [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: <class 'oslo_db.exception.DBReferenceError'> {{(pid=56087) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:943}}
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: INFO nova.api.openstack.requestlog [None req-19b3c8fc-586e-40ad-b3b7-3cc604632d69 tempest-DeleteServersTestJSON-1007295085 tempest-DeleteServersTestJSON-1007295085-project-member] 10.209.37.96 "DELETE /compute/v2.1/servers/61f3c713-8ef1-4281-a3db-f199197efda3" status: 500 len: 208 microversion: 2.1 time: 1.209063
May 03 20:10:51.149668 np0037436303 devstack@n-api.service[56087]: [pid: 56087|app: 0|req: 914/1850] 10.209.37.96 () {66 vars in 1379 bytes} [Fri May 3 20:10:49 2024] DELETE /compute/v2.1/servers/61f3c713-8ef1-4281-a3db-f199197efda3 => generated 208 bytes in 1211 msecs (HTTP/1.1 500) 9 headers in 391 bytes (1 switches on core 0)
Seen once in:-
https://b04549659f81127bccd5-1b102124c6fd0b6d71635117190cffc0.ssl.cf1.rackcdn.com/917262/2/check/neutron-ovs-tempest-multinode-full/a21ea6a/testr_results.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2065693/+subscriptions
