yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #90039
[Bug 1994102] [NEW] VM stuck in spawing state for 5 mins and then goes to error | Tripleo Wallaby
Public bug reported:
When i launch multiple VM using horizon like 4 or 5 instances almost all instances get created but 1 instance stays in spawining state and then after 5 mins goes to error.
Instance ID: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b
Logs:
[root@overcloud-novacompute-4 nova]# tail -f nova-compute.log
2022-10-25 04:28:46.238 7 INFO nova.compute.manager [-] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] VM Started (Lifecycle Event)
2022-10-25 04:28:46.244 7 INFO nova.virt.libvirt.driver [-] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Instance spawned successfully.
2022-10-25 04:28:46.245 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Took 3.97 seconds to spawn the instance on the hypervisor.
2022-10-25 04:28:46.320 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Took 15.98 seconds to build instance.
2022-10-25 04:28:46.347 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] VM Paused (Lifecycle Event)
2022-10-25 04:28:46.412 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] VM Started (Lifecycle Event)
2022-10-25 04:28:46.479 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] VM Paused (Lifecycle Event)
2022-10-25 04:28:46.509 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During sync_power_state the instance has a pending task (spawning). Skip.
2022-10-25 04:28:46.509 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] VM Resumed (Lifecycle Event)
2022-10-25 04:28:48.239 7 WARNING nova.compute.manager [req-f4572f97-d7b8-490b-869a-aeb870806bb9 9e91b69f45a049d9a682c0c2beaf4ff3 7c9659572662449da4906cc77b33f21b - default default] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Received unexpected event network-vif-plugged-cecc2cf5-bca1-48f8-8319-0c4bcd9f2ec5 for instance with vm_state active and task_state None.
2022-10-25 04:31:02.006 7 INFO nova.compute.manager [req-53408911-6523-4864-9402-2fc396ce467e c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Get console output
2022-10-25 04:31:02.011 7 INFO oslo.privsep.daemon [req-53408911-6523-4864-9402-2fc396ce467e c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova.conf', '--config-file', '/etc/nova/nova-compute.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmp0vz4g22a/privsep.sock']
2022-10-25 04:31:02.810 7 INFO oslo.privsep.daemon [req-53408911-6523-4864-9402-2fc396ce467e c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Spawned new privsep daemon via rootwrap
2022-10-25 04:31:02.724 714 INFO oslo.privsep.daemon [-] privsep daemon starting
2022-10-25 04:31:02.727 714 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2022-10-25 04:31:02.730 714 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
2022-10-25 04:31:02.730 714 INFO oslo.privsep.daemon [-] privsep daemon running as pid 714
2022-10-25 04:31:03.046 714 INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
2022-10-25 04:33:46.242 7 WARNING nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Timeout waiting for [('network-vif-plugged', '845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1')] for instance with vm_state building and task_state spawning: eventlet.timeout.Timeout: 300 seconds
2022-10-25 04:33:46.485 7 INFO os_vif [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6f:2c:94,bridge_name='br-int',has_traffic_filtering=True,id=845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1,network=Network(c01495d8-b3a8-4e32-843c-5b11608759a5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap845fe0fd-f3')
2022-10-25 04:33:46.487 7 INFO nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Deleting instance files /var/lib/nova/instances/65c1b043-4f6c-4d85-adb5-7223dbb21f4b_del
2022-10-25 04:33:46.488 7 INFO nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Deletion of /var/lib/nova/instances/65c1b043-4f6c-4d85-adb5-7223dbb21f4b_del complete
2022-10-25 04:33:46.660 7 INFO nova.virt.libvirt.host [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] UEFI support detected
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7274, in _create_guest_with_network
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] post_xml_callback=post_xml_callback)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] next(self.gen)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 481, in wait_for_instance_event
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] actual_event = event.wait()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] result = hub.switch()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 313, in switch
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] return self.greenlet.switch()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] eventlet.timeout.Timeout: 300 seconds
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During handling of the above exception, another exception occurred:
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2644, in _build_resources
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] yield resources
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2408, in _build_and_run_instance
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] accel_info=accel_info)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4230, in spawn
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] cleanup_instance_disks=created_disks)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7297, in _create_guest_with_network
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] raise exception.VirtualInterfaceCreateException()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:46.668 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Terminating instance
2022-10-25 04:33:46.676 7 INFO nova.virt.libvirt.driver [-] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Instance destroyed successfully.
2022-10-25 04:33:46.679 7 INFO os_vif [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6f:2c:94,bridge_name='br-int',has_traffic_filtering=True,id=845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1,network=Network(c01495d8-b3a8-4e32-843c-5b11608759a5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap845fe0fd-f3')
2022-10-25 04:33:47.244 7 INFO nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Deletion of /var/lib/nova/instances/65c1b043-4f6c-4d85-adb5-7223dbb21f4b_del complete
2022-10-25 04:33:47.324 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Took 0.65 seconds to destroy the instance on the hypervisor.
2022-10-25 04:33:48.018 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Took 0.69 seconds to detach 1 volumes for instance.
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7274, in _create_guest_with_network
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] post_xml_callback=post_xml_callback)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] next(self.gen)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 481, in wait_for_instance_event
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] actual_event = event.wait()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] result = hub.switch()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 313, in switch
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] return self.greenlet.switch()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] eventlet.timeout.Timeout: 300 seconds
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During handling of the above exception, another exception occurred:
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2408, in _build_and_run_instance
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] accel_info=accel_info)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4230, in spawn
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] cleanup_instance_disks=created_disks)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7297, in _create_guest_with_network
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] raise exception.VirtualInterfaceCreateException()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:48.961 7 ERROR nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Build of instance 65c1b043-4f6c-4d85-adb5-7223dbb21f4b aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 65c1b043-4f6c-4d85-adb5-7223dbb21f4b aborted: Failed to allocate the network(s), not rescheduling.
2022-10-25 04:33:48.966 7 INFO os_vif [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6f:2c:94,bridge_name='br-int',has_traffic_filtering=True,id=845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1,network=Network(c01495d8-b3a8-4e32-843c-5b11608759a5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap845fe0fd-f3')
2022-10-25 04:33:49.870 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Took 0.90 seconds to deallocate network for instance.
2022-10-25 04:33:49.916 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Detaching volume a0b2b53f-90a7-4994-9bf0-ff186acebd19
2022-10-25 04:33:50.002 7 ERROR nova.volume.cinder [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Delete attachment failed for attachment a5bb0bea-c822-42f6-954b-76d48a84a7f4. Error: Volume attachment could not be found with filter: attachment_id = a5bb0bea-c822-42f6-954b-76d48a84a7f4. (HTTP 404) (Request-ID: req-bae83121-76c0-4ae6-a8b0-2ba1a7d225a1) Code: 404: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = a5bb0bea-c822-42f6-954b-76d48a84a7f4. (HTTP 404) (Request-ID: req-bae83121-76c0-4ae6-a8b0-2ba1a7d225a1)
2022-10-25 04:33:50.003 7 INFO nova.virt.block_device [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Ignoring a volume attachment deletion failure as the volume a0b2b53f-90a7-4994-9bf0-ff186acebd19 or the volume attachment a5bb0bea-c822-42f6-954b-76d48a84a7f4 disappeared during the request.
2022-10-25 04:33:50.126 7 INFO nova.scheduler.client.report [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Deleted allocations for instance 65c1b043-4f6c-4d85-adb5-7223dbb21f4b
2022-10-25 04:33:50.127 7 INFO nova.compute.manager [-] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During sync_power_state the instance has a pending task (spawning). Skip.
2022-10-25 04:34:01.482 7 INFO nova.compute.manager [-] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] VM Stopped (Lifecycle Event)
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1994102
Title:
VM stuck in spawing state for 5 mins and then goes to error | Tripleo
Wallaby
Status in OpenStack Compute (nova):
New
Bug description:
When i launch multiple VM using horizon like 4 or 5 instances almost all instances get created but 1 instance stays in spawining state and then after 5 mins goes to error.
Instance ID: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b
Logs:
[root@overcloud-novacompute-4 nova]# tail -f nova-compute.log
2022-10-25 04:28:46.238 7 INFO nova.compute.manager [-] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] VM Started (Lifecycle Event)
2022-10-25 04:28:46.244 7 INFO nova.virt.libvirt.driver [-] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Instance spawned successfully.
2022-10-25 04:28:46.245 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Took 3.97 seconds to spawn the instance on the hypervisor.
2022-10-25 04:28:46.320 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Took 15.98 seconds to build instance.
2022-10-25 04:28:46.347 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] VM Paused (Lifecycle Event)
2022-10-25 04:28:46.412 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] VM Started (Lifecycle Event)
2022-10-25 04:28:46.479 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] VM Paused (Lifecycle Event)
2022-10-25 04:28:46.509 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During sync_power_state the instance has a pending task (spawning). Skip.
2022-10-25 04:28:46.509 7 INFO nova.compute.manager [req-c7d0f470-297d-4550-bf67-169a80bc79e8 - - - - -] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] VM Resumed (Lifecycle Event)
2022-10-25 04:28:48.239 7 WARNING nova.compute.manager [req-f4572f97-d7b8-490b-869a-aeb870806bb9 9e91b69f45a049d9a682c0c2beaf4ff3 7c9659572662449da4906cc77b33f21b - default default] [instance: db8d0266-6498-4e4a-adf4-f986878cdfc4] Received unexpected event network-vif-plugged-cecc2cf5-bca1-48f8-8319-0c4bcd9f2ec5 for instance with vm_state active and task_state None.
2022-10-25 04:31:02.006 7 INFO nova.compute.manager [req-53408911-6523-4864-9402-2fc396ce467e c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Get console output
2022-10-25 04:31:02.011 7 INFO oslo.privsep.daemon [req-53408911-6523-4864-9402-2fc396ce467e c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova.conf', '--config-file', '/etc/nova/nova-compute.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmp0vz4g22a/privsep.sock']
2022-10-25 04:31:02.810 7 INFO oslo.privsep.daemon [req-53408911-6523-4864-9402-2fc396ce467e c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Spawned new privsep daemon via rootwrap
2022-10-25 04:31:02.724 714 INFO oslo.privsep.daemon [-] privsep daemon starting
2022-10-25 04:31:02.727 714 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2022-10-25 04:31:02.730 714 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
2022-10-25 04:31:02.730 714 INFO oslo.privsep.daemon [-] privsep daemon running as pid 714
2022-10-25 04:31:03.046 714 INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
2022-10-25 04:33:46.242 7 WARNING nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Timeout waiting for [('network-vif-plugged', '845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1')] for instance with vm_state building and task_state spawning: eventlet.timeout.Timeout: 300 seconds
2022-10-25 04:33:46.485 7 INFO os_vif [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6f:2c:94,bridge_name='br-int',has_traffic_filtering=True,id=845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1,network=Network(c01495d8-b3a8-4e32-843c-5b11608759a5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap845fe0fd-f3')
2022-10-25 04:33:46.487 7 INFO nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Deleting instance files /var/lib/nova/instances/65c1b043-4f6c-4d85-adb5-7223dbb21f4b_del
2022-10-25 04:33:46.488 7 INFO nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Deletion of /var/lib/nova/instances/65c1b043-4f6c-4d85-adb5-7223dbb21f4b_del complete
2022-10-25 04:33:46.660 7 INFO nova.virt.libvirt.host [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] UEFI support detected
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Instance failed to spawn: nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7274, in _create_guest_with_network
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] post_xml_callback=post_xml_callback)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] next(self.gen)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 481, in wait_for_instance_event
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] actual_event = event.wait()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] result = hub.switch()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 313, in switch
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] return self.greenlet.switch()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] eventlet.timeout.Timeout: 300 seconds
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During handling of the above exception, another exception occurred:
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2644, in _build_resources
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] yield resources
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2408, in _build_and_run_instance
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] accel_info=accel_info)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4230, in spawn
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] cleanup_instance_disks=created_disks)
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7297, in _create_guest_with_network
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] raise exception.VirtualInterfaceCreateException()
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:46.662 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:46.668 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Terminating instance
2022-10-25 04:33:46.676 7 INFO nova.virt.libvirt.driver [-] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Instance destroyed successfully.
2022-10-25 04:33:46.679 7 INFO os_vif [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6f:2c:94,bridge_name='br-int',has_traffic_filtering=True,id=845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1,network=Network(c01495d8-b3a8-4e32-843c-5b11608759a5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap845fe0fd-f3')
2022-10-25 04:33:47.244 7 INFO nova.virt.libvirt.driver [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Deletion of /var/lib/nova/instances/65c1b043-4f6c-4d85-adb5-7223dbb21f4b_del complete
2022-10-25 04:33:47.324 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Took 0.65 seconds to destroy the instance on the hypervisor.
2022-10-25 04:33:48.018 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Took 0.69 seconds to detach 1 volumes for instance.
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7274, in _create_guest_with_network
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] post_xml_callback=post_xml_callback)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] next(self.gen)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 481, in wait_for_instance_event
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] actual_event = event.wait()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] result = hub.switch()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 313, in switch
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] return self.greenlet.switch()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] eventlet.timeout.Timeout: 300 seconds
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During handling of the above exception, another exception occurred:
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Traceback (most recent call last):
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2408, in _build_and_run_instance
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] accel_info=accel_info)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4230, in spawn
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] cleanup_instance_disks=created_disks)
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7297, in _create_guest_with_network
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] raise exception.VirtualInterfaceCreateException()
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed
2022-10-25 04:33:48.949 7 ERROR nova.compute.manager [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b]
2022-10-25 04:33:48.961 7 ERROR nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Build of instance 65c1b043-4f6c-4d85-adb5-7223dbb21f4b aborted: Failed to allocate the network(s), not rescheduling.: nova.exception.BuildAbortException: Build of instance 65c1b043-4f6c-4d85-adb5-7223dbb21f4b aborted: Failed to allocate the network(s), not rescheduling.
2022-10-25 04:33:48.966 7 INFO os_vif [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6f:2c:94,bridge_name='br-int',has_traffic_filtering=True,id=845fe0fd-f3d0-4b31-b030-d1f1f22e1aa1,network=Network(c01495d8-b3a8-4e32-843c-5b11608759a5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap845fe0fd-f3')
2022-10-25 04:33:49.870 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Took 0.90 seconds to deallocate network for instance.
2022-10-25 04:33:49.916 7 INFO nova.compute.manager [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] Detaching volume a0b2b53f-90a7-4994-9bf0-ff186acebd19
2022-10-25 04:33:50.002 7 ERROR nova.volume.cinder [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Delete attachment failed for attachment a5bb0bea-c822-42f6-954b-76d48a84a7f4. Error: Volume attachment could not be found with filter: attachment_id = a5bb0bea-c822-42f6-954b-76d48a84a7f4. (HTTP 404) (Request-ID: req-bae83121-76c0-4ae6-a8b0-2ba1a7d225a1) Code: 404: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = a5bb0bea-c822-42f6-954b-76d48a84a7f4. (HTTP 404) (Request-ID: req-bae83121-76c0-4ae6-a8b0-2ba1a7d225a1)
2022-10-25 04:33:50.003 7 INFO nova.virt.block_device [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Ignoring a volume attachment deletion failure as the volume a0b2b53f-90a7-4994-9bf0-ff186acebd19 or the volume attachment a5bb0bea-c822-42f6-954b-76d48a84a7f4 disappeared during the request.
2022-10-25 04:33:50.126 7 INFO nova.scheduler.client.report [req-14d5dc11-7b98-47a1-a382-f426b86feb79 c1905e3c5a374924980e851840e7f028 5d922243077045c48fe4b075e386551b - default default] Deleted allocations for instance 65c1b043-4f6c-4d85-adb5-7223dbb21f4b
2022-10-25 04:33:50.127 7 INFO nova.compute.manager [-] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] During sync_power_state the instance has a pending task (spawning). Skip.
2022-10-25 04:34:01.482 7 INFO nova.compute.manager [-] [instance: 65c1b043-4f6c-4d85-adb5-7223dbb21f4b] VM Stopped (Lifecycle Event)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1994102/+subscriptions
Follow ups