yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1270332] [NEW] cold migration fails in VMware driver

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sabari Kumar Murugesan <smurugesan@xxxxxxxxxx>
Date: Sat, 18 Jan 2014 00:29:01 -0000
Reply-to: Bug 1270332 <1270332@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

With two compute nodes (on different hosts) configured to two different
clusters in the same vCenter Server i.e :-


nova migrate <server> fails to migrate a server with the following error :-

2014-01-17 16:00:21.336 ERROR nova.openstack.common.rpc.amqp [req-0c587eb7-3a23-4790-b23d-f4ad005b5fe7 admin admin] Exception during message handling
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     **args)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/exception.py", line 90, in wrapped
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     payload)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/exception.py", line 73, in wrapped
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 244, in decorated_function
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     pass
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 230, in decorated_function
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     function(self, context, *args, **kwargs)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 272, in decorated_function
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 259, in decorated_function
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 3163, in resize_instance
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     block_device_info)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 406, in migrate_disk_and_power_off
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     dest, flavor)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1182, in migrate_disk_and_power_off
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     self._session._wait_for_task(instance['uuid'], vm_clone_task)
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 857, in _wait_for_task
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     ret_val = done.wait()
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return hubs.get_hub().switch()
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return self.greenlet.switch()
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp NovaException: Unable to access the virtual machine configuration: Unable to access file [datastore1]
2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp 
2014-01-17 16:00:21.497 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... from (pid=28144) multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:553
2014-01-17 16:00:21.498 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is b52829e908144155970ef23754eb46e3 from (pid=28144) multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:556
2014-01-17 16:00:21.498 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is 393d7230809b4892a4d7834c5ab56c2f. from (pid=28144) _add_unique_id /opt/stack/nova/nova/openstack/common/rpc/amqp.py:341
2014-01-17 16:00:25.640 WARNING nova.virt.vmwareapi.driver [-] Task [CloneVM_Task] (returnval){
   value = "task-110"
   _type = "Task"
 } status: error Unable to access the virtual machine configuration: Unable to access file [datastore1]

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: vmware

** Tags added: vmware

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1270332

Title:
  cold migration fails in VMware driver

Status in OpenStack Compute (Nova):
  New

Bug description:
  With two compute nodes (on different hosts) configured to two
  different clusters in the same vCenter Server i.e :-

  
  nova migrate <server> fails to migrate a server with the following error :-

  2014-01-17 16:00:21.336 ERROR nova.openstack.common.rpc.amqp [req-0c587eb7-3a23-4790-b23d-f4ad005b5fe7 admin admin] Exception during message handling
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     **args)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/exception.py", line 90, in wrapped
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     payload)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/exception.py", line 73, in wrapped
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 244, in decorated_function
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     pass
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 230, in decorated_function
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     function(self, context, *args, **kwargs)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 272, in decorated_function
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     e, sys.exc_info())
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     six.reraise(self.type_, self.value, self.tb)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 259, in decorated_function
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return function(self, context, *args, **kwargs)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/compute/manager.py", line 3163, in resize_instance
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     block_device_info)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 406, in migrate_disk_and_power_off
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     dest, flavor)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1182, in migrate_disk_and_power_off
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     self._session._wait_for_task(instance['uuid'], vm_clone_task)
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 857, in _wait_for_task
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     ret_val = done.wait()
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return hubs.get_hub().switch()
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp     return self.greenlet.switch()
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp NovaException: Unable to access the virtual machine configuration: Unable to access file [datastore1]
  2014-01-17 16:00:21.336 TRACE nova.openstack.common.rpc.amqp 
  2014-01-17 16:00:21.497 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... from (pid=28144) multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:553
  2014-01-17 16:00:21.498 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is b52829e908144155970ef23754eb46e3 from (pid=28144) multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:556
  2014-01-17 16:00:21.498 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is 393d7230809b4892a4d7834c5ab56c2f. from (pid=28144) _add_unique_id /opt/stack/nova/nova/openstack/common/rpc/amqp.py:341
  2014-01-17 16:00:25.640 WARNING nova.virt.vmwareapi.driver [-] Task [CloneVM_Task] (returnval){
     value = "task-110"
     _type = "Task"
   } status: error Unable to access the virtual machine configuration: Unable to access file [datastore1]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1270332/+subscriptions
Follow ups

[Bug 1270332] Re: cold migration fails in VMware driver
From: Markus Zoeller (markus_z), 2016-07-08
[Bug 1270332] [NEW] cold migration fails in VMware driver
From: Sabari Kumar Murugesan, 2014-01-18
References

[Bug 1270332] [NEW] cold migration fails in VMware driver
From: Sabari Kumar Murugesan, 2014-01-18