yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1649793] [NEW] vmware image download error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: YuYang <wangyy@xxxxxxxxxxx>
Date: Wed, 14 Dec 2016 08:31:48 -0000
Reply-to: Bug 1649793 <1649793@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

when create vm in openstack success,then i task a snapshot from the vm
,and then use the snapshot to deploy vm error.

i add some log in the code ,found write_handle only transfer several
times ,will break off.

the error msg:

2016-12-14 03:37:20.138 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:20.140 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:20.142 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:20.144 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:20.464 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:21.811 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:42.449 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:42.451 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
2016-12-14 03:37:42.508 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Getting lease state for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:459
2016-12-14 03:37:42.509 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] --------------start close--------------------- close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:460
2016-12-14 03:37:42.510 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] [(<frame object at 0x867d9a0>, '/usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py', 461, 'close', ['        LOG.debug(inspect.stack())\n'], 0), (<frame object at 0x8c47530>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py', 208, 'image_transfer', ['        write_handle.close()\n'], 0), (<frame object at 0x9781ee0>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py', 359, 'fetch_image_stream_optimized', ['    image_transfer(read_handle, write_handle)\n'], 0), (<frame object at 0x9816130>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py', 429, '_fetch_image_as_vapp', ['            self._root_resource_pool)\n'], 0), (<frame object at 0x984e870>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py', 614, '_fetch_image_if_missing', ['                image_fetch(context, vi, tmp_image_ds_loc)\n'], 0), (<frame object at 0x8e2a1c0>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py', 751, 'spawn', ['            self._fetch_image_if_missing(context, vi)\n'], 0), (<frame object at 0x8e29050>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py', 381, 'spawn', ['                          admin_password, network_info, block_device_info)\n'], 0), (<frame object at 0x9603b40>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 2064, '_build_and_run_instance', ['                                          block_device_info=block_device_info)\n'], 0), (<frame object at 0x966b050>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 1926, '_do_build_and_run_instance', ['                        filter_properties)\n'], 0), (<frame object at 0x962b420>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 375, 'decorated_function', ['            return function(self, context, *args, **kwargs)\n'], 0), (<frame object at 0x9672c40>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 409, 'decorated_function', ['            return function(self, context, *args, **kwargs)\n'], 0), (<frame object at 0x9063ef0>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 328, 'decorated_function', ['            return function(self, context, *args, **kwargs)\n'], 0), (<frame object at 0x8e2b2a0>, '/usr/lib/python2.7/dist-packages/nova/exception.py', 89, 'wrapped', ['                return f(self, context, *args, **kw)\n'], 0), (<frame object at 0x8e23360>, '/usr/lib/python2.7/dist-packages/nova/hooks.py', 154, 'inner', ['            rv = f(*args, **kwargs)\n'], 0), (<frame object at 0x7f30111faa00>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 1874, '_locked_do_build_and_run_instance', ['                self._do_build_and_run_instance(*args, **kwargs)\n'], 0), (<frame object at 0x9572860>, '/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py', 271, 'inner', ['                    return f(*args, **kwargs)\n'], 0), (<frame object at 0x7f3011200400>, '/usr/lib/python2.7/dist-packages/nova/utils.py', 1168, 'context_wrapper', ['        func(*args, **kwargs)\n'], 0)] close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:461
2016-12-14 03:37:42.511 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] --------------end close--------------------- close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:462
2016-12-14 03:37:42.512 26646 DEBUG oslo_vmware.api [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Waiting for function oslo_vmware.api._invoke_api to return. func /usr/lib/python2.7/dist-packages/oslo_vmware/api.py:122
2016-12-14 03:37:42.558 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Lease for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk is in state: ready. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:471
2016-12-14 03:37:42.558 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Releasing lease for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:473
2016-12-14 03:37:42.559 26646 DEBUG oslo_vmware.api [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Waiting for function oslo_vmware.api._invoke_api to return. func /usr/lib/python2.7/dist-packages/oslo_vmware/api.py:122
2016-12-14 03:37:44.453 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Closed VMDK write handle for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:488
2016-12-14 03:37:44.453 26646 DEBUG oslo_concurrency.lockutils [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Releasing semaphore "[WYYVSANDATASTOR] vcenter-WyyCluster_base/90d1d876-dff9-4385-a5c9-898b51b68ba0/90d1d876-dff9-4385-a5c9-898b51b68ba0.vmdk" lock /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:225
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c] Instance failed to spawn
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c] Traceback (most recent call last):
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2218, in _build_resources
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     yield resources
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2064, in _build_and_run_instance
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     block_device_info=block_device_info)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 381, in spawn
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     admin_password, network_info, block_device_info)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 751, in spawn
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     self._fetch_image_if_missing(context, vi)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 614, in _fetch_image_if_missing
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     image_fetch(context, vi, tmp_image_ds_loc)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 429, in _fetch_image_as_vapp
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     self._root_resource_pool)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py", line 359, in fetch_image_stream_optimized
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     image_transfer(read_handle, write_handle)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py", line 194, in image_transfer
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     data = read_handle.read(CHUNK_SIZE)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 656, in read
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     data = next(self._iter)
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 664, in get_next
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     for data in self._glance_read_iter:
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 477, in __iter__
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     for chunk in self.iterable:
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 435, in integrity_iter
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     (md5sum, checksum))
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c] IOError: [Errno 32] Corrupt image download. Checksum was 87ca5d11633f74c72d617c675ddc876c expected 18d4f2c38e8f1e9b709088139f77ba12
2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1649793

Title:
  vmware image download error

Status in OpenStack Compute (nova):
  New

Bug description:
  when create vm in openstack success,then i task a snapshot from the vm
  ,and then use the snapshot to deploy vm error.

  i add some log in the code ,found write_handle only transfer several
  times ,will break off.

  the error msg:

  2016-12-14 03:37:20.138 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:20.140 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:20.142 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:20.144 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:20.464 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:21.811 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:42.449 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:42.451 26646 ERROR nova.virt.vmwareapi.images [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] ---------------write_handle.write 1---------------
  2016-12-14 03:37:42.508 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Getting lease state for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:459
  2016-12-14 03:37:42.509 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] --------------start close--------------------- close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:460
  2016-12-14 03:37:42.510 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] [(<frame object at 0x867d9a0>, '/usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py', 461, 'close', ['        LOG.debug(inspect.stack())\n'], 0), (<frame object at 0x8c47530>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py', 208, 'image_transfer', ['        write_handle.close()\n'], 0), (<frame object at 0x9781ee0>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py', 359, 'fetch_image_stream_optimized', ['    image_transfer(read_handle, write_handle)\n'], 0), (<frame object at 0x9816130>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py', 429, '_fetch_image_as_vapp', ['            self._root_resource_pool)\n'], 0), (<frame object at 0x984e870>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py', 614, '_fetch_image_if_missing', ['                image_fetch(context, vi, tmp_image_ds_loc)\n'], 0), (<frame object at 0x8e2a1c0>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py', 751, 'spawn', ['            self._fetch_image_if_missing(context, vi)\n'], 0), (<frame object at 0x8e29050>, '/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py', 381, 'spawn', ['                          admin_password, network_info, block_device_info)\n'], 0), (<frame object at 0x9603b40>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 2064, '_build_and_run_instance', ['                                          block_device_info=block_device_info)\n'], 0), (<frame object at 0x966b050>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 1926, '_do_build_and_run_instance', ['                        filter_properties)\n'], 0), (<frame object at 0x962b420>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 375, 'decorated_function', ['            return function(self, context, *args, **kwargs)\n'], 0), (<frame object at 0x9672c40>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 409, 'decorated_function', ['            return function(self, context, *args, **kwargs)\n'], 0), (<frame object at 0x9063ef0>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 328, 'decorated_function', ['            return function(self, context, *args, **kwargs)\n'], 0), (<frame object at 0x8e2b2a0>, '/usr/lib/python2.7/dist-packages/nova/exception.py', 89, 'wrapped', ['                return f(self, context, *args, **kw)\n'], 0), (<frame object at 0x8e23360>, '/usr/lib/python2.7/dist-packages/nova/hooks.py', 154, 'inner', ['            rv = f(*args, **kwargs)\n'], 0), (<frame object at 0x7f30111faa00>, '/usr/lib/python2.7/dist-packages/nova/compute/manager.py', 1874, '_locked_do_build_and_run_instance', ['                self._do_build_and_run_instance(*args, **kwargs)\n'], 0), (<frame object at 0x9572860>, '/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py', 271, 'inner', ['                    return f(*args, **kwargs)\n'], 0), (<frame object at 0x7f3011200400>, '/usr/lib/python2.7/dist-packages/nova/utils.py', 1168, 'context_wrapper', ['        func(*args, **kwargs)\n'], 0)] close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:461
  2016-12-14 03:37:42.511 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] --------------end close--------------------- close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:462
  2016-12-14 03:37:42.512 26646 DEBUG oslo_vmware.api [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Waiting for function oslo_vmware.api._invoke_api to return. func /usr/lib/python2.7/dist-packages/oslo_vmware/api.py:122
  2016-12-14 03:37:42.558 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Lease for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk is in state: ready. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:471
  2016-12-14 03:37:42.558 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Releasing lease for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:473
  2016-12-14 03:37:42.559 26646 DEBUG oslo_vmware.api [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Waiting for function oslo_vmware.api._invoke_api to return. func /usr/lib/python2.7/dist-packages/oslo_vmware/api.py:122
  2016-12-14 03:37:44.453 26646 DEBUG oslo_vmware.rw_handles [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Closed VMDK write handle for https://192.168.103.86/nfc/52c6f60a-50c4-e232-7659-b0927f1dd866/disk-0.vmdk. close /usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py:488
  2016-12-14 03:37:44.453 26646 DEBUG oslo_concurrency.lockutils [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] Releasing semaphore "[WYYVSANDATASTOR] vcenter-WyyCluster_base/90d1d876-dff9-4385-a5c9-898b51b68ba0/90d1d876-dff9-4385-a5c9-898b51b68ba0.vmdk" lock /usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:225
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [req-d2899c45-97fc-4077-8b9d-b8152532f7c5 098673ebb2c54b6ea608b204ffaa82b7 3885b586a8014286bc07b24df332afab - - -] [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c] Instance failed to spawn
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c] Traceback (most recent call last):
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2218, in _build_resources
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     yield resources
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2064, in _build_and_run_instance
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     block_device_info=block_device_info)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/driver.py", line 381, in spawn
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     admin_password, network_info, block_device_info)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 751, in spawn
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     self._fetch_image_if_missing(context, vi)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 614, in _fetch_image_if_missing
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     image_fetch(context, vi, tmp_image_ds_loc)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/vmops.py", line 429, in _fetch_image_as_vapp
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     self._root_resource_pool)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py", line 359, in fetch_image_stream_optimized
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     image_transfer(read_handle, write_handle)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/nova/virt/vmwareapi/images.py", line 194, in image_transfer
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     data = read_handle.read(CHUNK_SIZE)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 656, in read
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     data = next(self._iter)
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/oslo_vmware/rw_handles.py", line 664, in get_next
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     for data in self._glance_read_iter:
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 477, in __iter__
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     for chunk in self.iterable:
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]   File "/usr/lib/python2.7/dist-packages/glanceclient/common/utils.py", line 435, in integrity_iter
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]     (md5sum, checksum))
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c] IOError: [Errno 32] Corrupt image download. Checksum was 87ca5d11633f74c72d617c675ddc876c expected 18d4f2c38e8f1e9b709088139f77ba12
  2016-12-14 03:37:44.454 26646 ERROR nova.compute.manager [instance: 484c1bcc-216c-4f8a-98ad-15d96f09a38c]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1649793/+subscriptions
Follow ups

[Bug 1649793] Re: vmware image download error
From: YuYang, 2017-01-11