yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #75199
[Bug 1797115] [NEW] Instance data deleted on shared storage after failed evacuate (queens)
Public bug reported:
Hi
The issue was experienced after one of compute nodes crashed. Then, after reboot of compute node, few instances was restarted, but rest of instances failed to start with error:
"Failed to get "write" lock
Is another process using the image?"
I tried to evacuate one instance to another host:
nova evacuate instance host2
After that nova compute on destination host throwed the same error:
Setting instance vm_state to ERROR: libvirtError: internal error: process exited while connecting to monitor: 2018-10-10T10:16:09.816477Z qemu-system-x86_64: -drive file=/var/lib/nova/instances/ed7760a8-3008-4feb-83f3-3b753b0e7d6e/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Failed to get "write" lock
Is another process using the image?
Regardless of reason of this issue, the whole directory of the instance
was gone from shared nfs, so /var/lib/nova/instances/ed7760a8-3008-4feb-
83f3-3b753b0e7d6e was removed with instance disk.
Then I reproduced the error. After backup of disk file, evacuate and
restore instance directory with disk file, the instance was started
successfully.
** Affects: nova
Importance: Undecided
Status: New
** Tags: evacuate
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1797115
Title:
Instance data deleted on shared storage after failed evacuate (queens)
Status in OpenStack Compute (nova):
New
Bug description:
Hi
The issue was experienced after one of compute nodes crashed. Then, after reboot of compute node, few instances was restarted, but rest of instances failed to start with error:
"Failed to get "write" lock
Is another process using the image?"
I tried to evacuate one instance to another host:
nova evacuate instance host2
After that nova compute on destination host throwed the same error:
Setting instance vm_state to ERROR: libvirtError: internal error: process exited while connecting to monitor: 2018-10-10T10:16:09.816477Z qemu-system-x86_64: -drive file=/var/lib/nova/instances/ed7760a8-3008-4feb-83f3-3b753b0e7d6e/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Failed to get "write" lock
Is another process using the image?
Regardless of reason of this issue, the whole directory of the
instance was gone from shared nfs, so
/var/lib/nova/instances/ed7760a8-3008-4feb-83f3-3b753b0e7d6e was
removed with instance disk.
Then I reproduced the error. After backup of disk file, evacuate and
restore instance directory with disk file, the instance was started
successfully.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1797115/+subscriptions