← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #48754

[Bug 1563684] [NEW] VMware: rebuild an instance failed

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Description
===================
Nova version: master
Virt driver: VCDriver


Steps to reproduce
=====================
1. Nova boot an instance.
2. Cinder create a volume.
3. Attach volume to the instance.
4. After attached success,  using nova image-create command, create a snapshot for the instance.
5. Rebuild the instance with snapshot image.

In nova-compute, error info:
2016-03-30 12:16:14.801 9290 DEBUG oslo_vmware.exceptions [-] Fault GenericVmConfigFault not matched. get_fault_class /usr/lib/python2.7/site-packages/oslo_vmware/exceptions.py:250
2016-03-30 12:16:14.801 9290 ERROR oslo_vmware.common.loopingcall [-] in fixed duration looping call
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall Traceback (most recent call last):
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall   File "/usr/lib/python2.7/site-packages/oslo_vmware/common/loopingcall.py", line 76, in _inner
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall     self.f(*self.args, **self.kw)
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall   File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 417, in _poll_task
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall     raise task_ex
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall VMwareDriverException: Disk '/vmfs/volumes/546a3532-ca1f27f1-e66a-1458d04cf670/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3.vmdk' cannot be opened for writing. It might be shared with some other VM.
2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall
2016-03-30 12:16:14.802 9290 ERROR nova.compute.manager [req-b0c16cc2-fd52-4c85-9ff9-c67ff68ee410 4412e38ec9814b96a03e63097ec51f1a 8f75187cd29f4715881f450646fc6e08 - - -] [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05] Setting instance vm_state to ERROR
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05] Traceback (most recent call last):
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6418, in _error_out_instance_on_exception
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     yield
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3087, in rebuild_instance
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     self._rebuild_default_impl(**kwargs)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2934, in _rebuild_default_impl
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     block_device_info=new_block_device_info)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 481, in spawn
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     admin_password, network_info, block_device_info)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vmops.py", line 683, in spawn
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     vm_util.power_on_instance(self._session, instance, vm_ref=vm_ref)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vm_util.py", line 1394, in power_on_instance
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     session._wait_for_task(poweron_task)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 681, in _wait_for_task
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return self.wait_for_task(task_ref)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 380, in wait_for_task
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return evt.wait()
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return hubs.get_hub().switch()
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return self.greenlet.switch()
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/oslo_vmware/common/loopingcall.py", line 76, in _inner
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     self.f(*self.args, **self.kw)
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 417, in _poll_task
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     raise task_ex
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05] VMwareDriverException: Disk '/vmfs/volumes/546a3532-ca1f27f1-e66a-1458d04cf670/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3.vmdk' cannot be opened for writing. It might be shared with some other VM.
2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: vmware

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1563684

Title:
  VMware: rebuild an instance failed

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===================
  Nova version: master
  Virt driver: VCDriver


  Steps to reproduce
  =====================
  1. Nova boot an instance.
  2. Cinder create a volume.
  3. Attach volume to the instance.
  4. After attached success,  using nova image-create command, create a snapshot for the instance.
  5. Rebuild the instance with snapshot image.

  In nova-compute, error info:
  2016-03-30 12:16:14.801 9290 DEBUG oslo_vmware.exceptions [-] Fault GenericVmConfigFault not matched. get_fault_class /usr/lib/python2.7/site-packages/oslo_vmware/exceptions.py:250
  2016-03-30 12:16:14.801 9290 ERROR oslo_vmware.common.loopingcall [-] in fixed duration looping call
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall Traceback (most recent call last):
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall   File "/usr/lib/python2.7/site-packages/oslo_vmware/common/loopingcall.py", line 76, in _inner
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall     self.f(*self.args, **self.kw)
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall   File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 417, in _poll_task
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall     raise task_ex
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall VMwareDriverException: Disk '/vmfs/volumes/546a3532-ca1f27f1-e66a-1458d04cf670/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3.vmdk' cannot be opened for writing. It might be shared with some other VM.
  2016-03-30 12:16:14.801 9290 TRACE oslo_vmware.common.loopingcall
  2016-03-30 12:16:14.802 9290 ERROR nova.compute.manager [req-b0c16cc2-fd52-4c85-9ff9-c67ff68ee410 4412e38ec9814b96a03e63097ec51f1a 8f75187cd29f4715881f450646fc6e08 - - -] [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05] Setting instance vm_state to ERROR
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05] Traceback (most recent call last):
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6418, in _error_out_instance_on_exception
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     yield
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3087, in rebuild_instance
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     self._rebuild_default_impl(**kwargs)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2934, in _rebuild_default_impl
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     block_device_info=new_block_device_info)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 481, in spawn
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     admin_password, network_info, block_device_info)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vmops.py", line 683, in spawn
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     vm_util.power_on_instance(self._session, instance, vm_ref=vm_ref)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/vm_util.py", line 1394, in power_on_instance
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     session._wait_for_task(poweron_task)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/nova/virt/vmwareapi/driver.py", line 681, in _wait_for_task
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return self.wait_for_task(task_ref)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 380, in wait_for_task
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return evt.wait()
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 121, in wait
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return hubs.get_hub().switch()
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     return self.greenlet.switch()
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/oslo_vmware/common/loopingcall.py", line 76, in _inner
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     self.f(*self.args, **self.kw)
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]   File "/usr/lib/python2.7/site-packages/oslo_vmware/api.py", line 417, in _poll_task
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]     raise task_ex
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05] VMwareDriverException: Disk '/vmfs/volumes/546a3532-ca1f27f1-e66a-1458d04cf670/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3/volume-dbbdd17d-bb5c-4804-9d30-a6bd8a96eca3.vmdk' cannot be opened for writing. It might be shared with some other VM.
  2016-03-30 12:16:14.802 9290 TRACE nova.compute.manager [instance: e338fd04-859f-4fa4-8f1d-cd2e297b0c05]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1563684/+subscriptions
  Thread Previous • Date Previous • • Thread Next