← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #31147

[Bug 1441235] [NEW] VMware: Fail to take a snapshot of a VM

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

With the latest code, cluster as  the compute driver, Taking a  snapshot
is failing with following error :

ERROR oslo_messaging.rpc.dispatcher [req-9c49bfdd-641c-4022-90cc-aae697e387f3 admin admin] Exception during message handling: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     payload)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 324, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance_uuid=instance_uuid)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 352, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 340, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 400, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     instance=instance)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 390, in decorated_function
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     *args, **kwargs)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 3201, in snapshot_instance
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     task_states.IMAGE_SNAPSHOT)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 3231, in _snapshot_instance
2015-04-07 2

1:16:47.166 TRACE oslo_messaging.rpc.dispatcher     update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 506, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     self._vmops.snapshot(context, instance, image_id, update_task_state)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 833, in snapshot
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     vmdk_size=vmdk.capacity_in_bytes)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/images.py", line 457, in upload_image_stream_optimized
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     vmdk_size)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 500, in __init__
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     cacerts=cacerts)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 79, in _create_read_connection
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     raise exceptions.VimException(excep_msg, excep)
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher VimException: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher

git log -1 : 
commit bf70df295b0529da9f4381f52fc94d328fa2fdb3
Merge: ad329b0 7560620
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date:   Mon Apr 6 23:57:10 2015 +0000

    Merge "neutronv2: only create client once when adding/removing fixed
IPs"


Environment : VC 5.5 U1, ESXi 5.5.0, 1834933


Is this an environment issue ? Export OVS template task is stuck for a long time and it fails, attached the snapshot.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: vmware

** Attachment added: "Capture-snaps.PNG"
   https://bugs.launchpad.net/bugs/1441235/+attachment/4368597/+files/Capture-snaps.PNG

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1441235

Title:
  VMware: Fail to take a snapshot of a VM

Status in OpenStack Compute (Nova):
  New

Bug description:
  With the latest code, cluster as  the compute driver, Taking a
  snapshot is failing with following error :

  ERROR oslo_messaging.rpc.dispatcher [req-9c49bfdd-641c-4022-90cc-aae697e387f3 admin admin] Exception during message handling: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
  Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     payload)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 324, in decorated_function
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance_uuid=instance_uuid)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 295, in decorated_function
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 352, in decorated_function
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 340, in decorated_function
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 400, in decorated_function
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     instance=instance)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 390, in decorated_function
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     *args, **kwargs)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 3201, in snapshot_instance
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     task_states.IMAGE_SNAPSHOT)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 3231, in _snapshot_instance
  2015-04-07 2

  1:16:47.166 TRACE oslo_messaging.rpc.dispatcher     update_task_state)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 506, in snapshot
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     self._vmops.snapshot(context, instance, image_id, update_task_state)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 833, in snapshot
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     vmdk_size=vmdk.capacity_in_bytes)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/vmwareapi/images.py", line 457, in upload_image_stream_optimized
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     vmdk_size)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 500, in __init__
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     cacerts=cacerts)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 79, in _create_read_connection
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher     raise exceptions.VimException(excep_msg, excep)
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher VimException: Error occurred while opening URL: https://10.1.214.7/nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk for reading.
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher Cause: HTTPSConnectionPool(host='10.1.214.7', port=443): Max retries exceeded with url: /nfc/52135ab3-7409-6a9c-30ba-65f00d264262/disk-1.vmdk (Caused by ProxyError('Cannot connect to proxy.', error('Tunnel connection failed: 503 Service Unavailable',)))
  2015-04-07 21:16:47.166 TRACE oslo_messaging.rpc.dispatcher

  git log -1 : 
  commit bf70df295b0529da9f4381f52fc94d328fa2fdb3
  Merge: ad329b0 7560620
  Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
  Date:   Mon Apr 6 23:57:10 2015 +0000

      Merge "neutronv2: only create client once when adding/removing
  fixed IPs"

  
  Environment : VC 5.5 U1, ESXi 5.5.0, 1834933

  
  Is this an environment issue ? Export OVS template task is stuck for a long time and it fails, attached the snapshot.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1441235/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next