yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2114730] [NEW] unshelve fails in test_cold_migrate_unshelved_instance tempest test with instance not found

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Balazs Gibizer <2114730@xxxxxxxxxxxxxxxxxx>
Date: Mon, 16 Jun 2025 12:15:50 -0000
Reply-to: Bug 2114730 <2114730@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Example failure:
https://zuul.opendev.org/t/openstack/build/c4a759f5a8ef41b6861d056a3190d8d5/logs

tempest:
Details: (TestShelveInstance:test_cold_migrate_unshelved_instance) Server 97a2b17a-eb7a-4587-87ac-f7768cdda5aa failed to reach ACTIVE status and task state "None" within the required time (196 s). Current status: SHELVED_OFFLOADED. Current task state: spawning.


nova-compute:

Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
61195	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 8241, in _create_guest_with_network
61196	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     with self.virtapi.wait_for_instance_event(
61197	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
61198	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     next(self.gen)
61199	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 560, in wait_for_instance_event
61200	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self._wait_for_instance_events(
61201	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 472, in _wait_for_instance_events
61202	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     actual_event = event.wait()
61203	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                    ^^^^^^^^^^^^
61204	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 437, in wait
61205	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     instance_event = self.event.wait()
61206	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                      ^^^^^^^^^^^^^^^^^
61207	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/eventlet/event.py", line 124, in wait
61208	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     result = hub.switch()
61209	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]              ^^^^^^^^^^^^
61210	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/eventlet/hubs/hub.py", line 310, in switch
61211	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return self.greenlet.switch()
61212	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^
61213	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] eventlet.timeout.Timeout: 300 seconds
61214	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61215	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] During handling of the above exception, another exception occurred:
61216	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61217	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
61218	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 7694, in _unshelve_instance
61219	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.driver.spawn(context, instance, image_meta,
61220	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4864, in spawn
61221	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self._create_guest_with_network(
61222	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 8267, in _create_guest_with_network
61223	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise exception.VirtualInterfaceCreateException()
61224	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
61225	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61226	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] During handling of the above exception, another exception occurred:
61227	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61228	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
61229	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 7686, in _unshelve_instance
61230	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     with self.rt.instance_claim(context, instance, node, allocations,
61231	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/claims.py", line 43, in __exit__
61232	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.abort()
61233	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/claims.py", line 86, in abort
61234	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.tracker.abort_instance_claim(self.context, self.instance_ref,
61235	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py", line 415, in inner
61236	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(*args, **kwargs)
61237	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^
61238	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 614, in abort_instance_claim
61239	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self._unset_instance_host_and_node(instance)
61240	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 605, in _unset_instance_host_and_node
61241	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     instance.save()
61242	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_versionedobjects/base.py", line 209, in wrapper
61243	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     updates, result = self.indirection_api.object_action(
61244	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61245	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/conductor/rpcapi.py", line 247, in object_action
61246	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return cctxt.call(context, 'object_action', objinst=objinst,
61247	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61248	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/rpc/client.py", line 180, in call
61249	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     result = self.transport._send(
61250	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]              ^^^^^^^^^^^^^^^^^^^^^
61251	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/transport.py", line 123, in _send
61252	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return self._driver.send(target, ctxt, message,
61253	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61254	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 800, in send
61255	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return self._send(target, ctxt, message, wait_for_reply, timeout,
61256	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61257	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 792, in _send
61258	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise result
61259	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] nova.exception_Remote.InstanceNotFound_Remote: Instance 97a2b17a-eb7a-4587-87ac-f7768cdda5aa could not be found.
61260	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
61261	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61262	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/conductor/manager.py", line 142, in _object_dispatch
61263	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return getattr(target, method)(*args, **kwargs)
61264	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61265	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61266	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_versionedobjects/base.py", line 226, in wrapper
61267	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return fn(self, *args, **kwargs)
61268	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^
61269	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61270	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/objects/instance.py", line 878, in save
61271	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     old_ref, inst_ref = db.instance_update_and_get_original(
61272	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61273	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61274	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper
61275	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(*args, **kwargs)
61276	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^
61277	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61278	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/api.py", line 144, in wrapper
61279	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     with excutils.save_and_reraise_exception() as ectxt:
61280	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61281	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 227, in __exit__
61282	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.force_reraise()
61283	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61284	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
61285	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise self.value
61286	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61287	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/api.py", line 142, in wrapper
61288	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(*args, **kwargs)
61289	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^
61290	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61291	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/main/api.py", line 207, in wrapper
61292	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(context, *args, **kwargs)
61293	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
61294	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61295	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/main/api.py", line 2294, in instance_update_and_get_original
61296	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     instance_ref = _instance_get_by_uuid(context, instance_uuid,
61297	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
61298	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61299	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/main/api.py", line 1404, in _instance_get_by_uuid
61300	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise exception.InstanceNotFound(instance_id=uuid)
61301	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61302	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] nova.exception.InstanceNotFound: Instance 97a2b17a-eb7a-4587-87ac-f7768cdda5aa could not be found.
61303	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
61304	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2114730

Title:
  unshelve fails in test_cold_migrate_unshelved_instance tempest test
  with instance not found

Status in OpenStack Compute (nova):
  New

Bug description:
  Example failure:
  https://zuul.opendev.org/t/openstack/build/c4a759f5a8ef41b6861d056a3190d8d5/logs

  tempest:
  Details: (TestShelveInstance:test_cold_migrate_unshelved_instance) Server 97a2b17a-eb7a-4587-87ac-f7768cdda5aa failed to reach ACTIVE status and task state "None" within the required time (196 s). Current status: SHELVED_OFFLOADED. Current task state: spawning.


  nova-compute:

  Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
  61195	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 8241, in _create_guest_with_network
  61196	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     with self.virtapi.wait_for_instance_event(
  61197	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
  61198	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     next(self.gen)
  61199	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 560, in wait_for_instance_event
  61200	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self._wait_for_instance_events(
  61201	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 472, in _wait_for_instance_events
  61202	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     actual_event = event.wait()
  61203	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                    ^^^^^^^^^^^^
  61204	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 437, in wait
  61205	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     instance_event = self.event.wait()
  61206	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                      ^^^^^^^^^^^^^^^^^
  61207	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/eventlet/event.py", line 124, in wait
  61208	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     result = hub.switch()
  61209	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]              ^^^^^^^^^^^^
  61210	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/eventlet/hubs/hub.py", line 310, in switch
  61211	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return self.greenlet.switch()
  61212	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^
  61213	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] eventlet.timeout.Timeout: 300 seconds
  61214	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61215	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] During handling of the above exception, another exception occurred:
  61216	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61217	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
  61218	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 7694, in _unshelve_instance
  61219	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.driver.spawn(context, instance, image_meta,
  61220	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4864, in spawn
  61221	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self._create_guest_with_network(
  61222	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 8267, in _create_guest_with_network
  61223	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise exception.VirtualInterfaceCreateException()
  61224	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
  61225	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61226	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] During handling of the above exception, another exception occurred:
  61227	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61228	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
  61229	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/manager.py", line 7686, in _unshelve_instance
  61230	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     with self.rt.instance_claim(context, instance, node, allocations,
  61231	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/claims.py", line 43, in __exit__
  61232	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.abort()
  61233	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/claims.py", line 86, in abort
  61234	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.tracker.abort_instance_claim(self.context, self.instance_ref,
  61235	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py", line 415, in inner
  61236	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(*args, **kwargs)
  61237	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^
  61238	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 614, in abort_instance_claim
  61239	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self._unset_instance_host_and_node(instance)
  61240	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/compute/resource_tracker.py", line 605, in _unset_instance_host_and_node
  61241	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     instance.save()
  61242	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_versionedobjects/base.py", line 209, in wrapper
  61243	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     updates, result = self.indirection_api.object_action(
  61244	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61245	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/conductor/rpcapi.py", line 247, in object_action
  61246	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return cctxt.call(context, 'object_action', objinst=objinst,
  61247	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61248	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/rpc/client.py", line 180, in call
  61249	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     result = self.transport._send(
  61250	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]              ^^^^^^^^^^^^^^^^^^^^^
  61251	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/transport.py", line 123, in _send
  61252	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return self._driver.send(target, ctxt, message,
  61253	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61254	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 800, in send
  61255	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return self._send(target, ctxt, message, wait_for_reply, timeout,
  61256	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61257	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 792, in _send
  61258	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise result
  61259	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] nova.exception_Remote.InstanceNotFound_Remote: Instance 97a2b17a-eb7a-4587-87ac-f7768cdda5aa could not be found.
  61260	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] Traceback (most recent call last):
  61261	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61262	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/conductor/manager.py", line 142, in _object_dispatch
  61263	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return getattr(target, method)(*args, **kwargs)
  61264	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61265	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61266	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_versionedobjects/base.py", line 226, in wrapper
  61267	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return fn(self, *args, **kwargs)
  61268	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^
  61269	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61270	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/objects/instance.py", line 878, in save
  61271	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     old_ref, inst_ref = db.instance_update_and_get_original(
  61272	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61273	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61274	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper
  61275	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(*args, **kwargs)
  61276	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^
  61277	Jun 13 15:44:53.985082 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61278	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/api.py", line 144, in wrapper
  61279	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     with excutils.save_and_reraise_exception() as ectxt:
  61280	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61281	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 227, in __exit__
  61282	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     self.force_reraise()
  61283	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61284	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
  61285	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise self.value
  61286	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61287	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/api.py", line 142, in wrapper
  61288	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(*args, **kwargs)
  61289	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^
  61290	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61291	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/main/api.py", line 207, in wrapper
  61292	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     return f(context, *args, **kwargs)
  61293	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61294	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61295	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/main/api.py", line 2294, in instance_update_and_get_original
  61296	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     instance_ref = _instance_get_by_uuid(context, instance_uuid,
  61297	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  61298	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61299	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]   File "/opt/stack/nova/nova/db/main/api.py", line 1404, in _instance_get_by_uuid
  61300	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]     raise exception.InstanceNotFound(instance_id=uuid)
  61301	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61302	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] nova.exception.InstanceNotFound: Instance 97a2b17a-eb7a-4587-87ac-f7768cdda5aa could not be found.
  61303	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa] 
  61304	Jun 13 15:44:53.992765 np0041119980 nova-compute[41689]: ERROR nova.compute.manager [instance: 97a2b17a-eb7a-4587-87ac-f7768cdda5aa]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2114730/+subscriptions