← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #94306

[Bug 2073944] [NEW] compute tests involving shelve/snapshots/rescue fails with Image is unacceptable: Image content does not match disk_format

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Happening since https://review.opendev.org/c/openstack/nova/+/924731 merged.
Fails as:-
Jul 24 03:01:11.545675 np0038048936 nova-compute[63573]: WARNING nova.virt.images [None req-40c52839-a391-4937-b739-50a64c1df107 tempest-ServerStableDeviceRescueTest-714156962 tempest-ServerStableDeviceRescueTest-714156962-project-member] Image b2f35142-576c-405f-b60b-f1ee77519977 detected by qemu as qcow2 but we expected raw
Jul 24 03:01:11.546569 np0038048936 nova-compute[63573]: DEBUG oslo_concurrency.lockutils [None req-40c52839-a391-4937-b739-50a64c1df107 tempest-ServerStableDeviceRescueTest-714156962 tempest-ServerStableDeviceRescueTest-714156962-project-member] Lock "b520937fa83f7b17fa2f7e0b4527570233398199" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.237s {{(pid=63573) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}}
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [None req-40c52839-a391-4937-b739-50a64c1df107 tempest-ServerStableDeviceRescueTest-714156962 tempest-ServerStableDeviceRescueTest-714156962-project-member] [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] Error trying to Rescue Instance: nova.exception.ImageUnacceptable: Image b2f35142-576c-405f-b60b-f1ee77519977 is unacceptable: Image content does not match disk_format
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] Traceback (most recent call last):
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/compute/manager.py", line 4709, in rescue_instance
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self.driver.rescue(context, instance, network_info,
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4500, in rescue
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self._create_image(context, instance, disk_info['mapping'],
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5104, in _create_image
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     created_disks = self._create_and_inject_local_root(
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5236, in _create_and_inject_local_root
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self._try_fetch_image_cache(backend, fetch_func, context,
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 11504, in _try_fetch_image_cache
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     image.cache(fetch_func=fetch_func,
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 301, in cache
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self.create_image(fetch_func_sync, base, size,
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 676, in create_image
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     prepare_template(target=base, *args, **kwargs)
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py", line 412, in inner
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     return f(*args, **kwargs)
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 298, in fetch_func_sync
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     fetch_func(target=target, *args, **kwargs)
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/utils.py", line 487, in fetch_image
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     images.fetch_to_raw(context, image_id, target, trusted_certs)
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/images.py", line 221, in fetch_to_raw
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     raise exception.ImageUnacceptable(
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] nova.exception.ImageUnacceptable: Image b2f35142-576c-405f-b60b-f1ee77519977 is unacceptable: Image content does not match disk_format
Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] 


example failures:-
- https://c5d689111c1590d08c48-aace7598638ebd17f7ac5b5df39ccb78.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-ovs-tempest-fips/bb19f26/testr_results.html
- https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_877/periodic/opendev.org/openstack/neutron/master/neutron-ovs-tempest-slow/8773a57/testr_results.html
- https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_746/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-with-neutron-lib-master/7460037/testr_results.html

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2073944

Title:
  compute tests involving shelve/snapshots/rescue fails with Image is
  unacceptable: Image content does not match disk_format

Status in OpenStack Compute (nova):
  New

Bug description:
  Happening since https://review.opendev.org/c/openstack/nova/+/924731 merged.
  Fails as:-
  Jul 24 03:01:11.545675 np0038048936 nova-compute[63573]: WARNING nova.virt.images [None req-40c52839-a391-4937-b739-50a64c1df107 tempest-ServerStableDeviceRescueTest-714156962 tempest-ServerStableDeviceRescueTest-714156962-project-member] Image b2f35142-576c-405f-b60b-f1ee77519977 detected by qemu as qcow2 but we expected raw
  Jul 24 03:01:11.546569 np0038048936 nova-compute[63573]: DEBUG oslo_concurrency.lockutils [None req-40c52839-a391-4937-b739-50a64c1df107 tempest-ServerStableDeviceRescueTest-714156962 tempest-ServerStableDeviceRescueTest-714156962-project-member] Lock "b520937fa83f7b17fa2f7e0b4527570233398199" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 0.237s {{(pid=63573) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:421}}
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [None req-40c52839-a391-4937-b739-50a64c1df107 tempest-ServerStableDeviceRescueTest-714156962 tempest-ServerStableDeviceRescueTest-714156962-project-member] [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] Error trying to Rescue Instance: nova.exception.ImageUnacceptable: Image b2f35142-576c-405f-b60b-f1ee77519977 is unacceptable: Image content does not match disk_format
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] Traceback (most recent call last):
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/compute/manager.py", line 4709, in rescue_instance
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self.driver.rescue(context, instance, network_info,
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4500, in rescue
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self._create_image(context, instance, disk_info['mapping'],
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5104, in _create_image
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     created_disks = self._create_and_inject_local_root(
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5236, in _create_and_inject_local_root
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self._try_fetch_image_cache(backend, fetch_func, context,
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 11504, in _try_fetch_image_cache
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     image.cache(fetch_func=fetch_func,
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 301, in cache
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     self.create_image(fetch_func_sync, base, size,
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 676, in create_image
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     prepare_template(target=base, *args, **kwargs)
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py", line 412, in inner
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     return f(*args, **kwargs)
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 298, in fetch_func_sync
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     fetch_func(target=target, *args, **kwargs)
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/libvirt/utils.py", line 487, in fetch_image
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     images.fetch_to_raw(context, image_id, target, trusted_certs)
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]   File "/opt/stack/nova/nova/virt/images.py", line 221, in fetch_to_raw
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8]     raise exception.ImageUnacceptable(
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] nova.exception.ImageUnacceptable: Image b2f35142-576c-405f-b60b-f1ee77519977 is unacceptable: Image content does not match disk_format
  Jul 24 03:01:11.547701 np0038048936 nova-compute[63573]: ERROR nova.compute.manager [instance: 4a8d02ce-aa9a-4318-bfb8-de91d7c6acf8] 

  
  example failures:-
  - https://c5d689111c1590d08c48-aace7598638ebd17f7ac5b5df39ccb78.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-ovs-tempest-fips/bb19f26/testr_results.html
  - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_877/periodic/opendev.org/openstack/neutron/master/neutron-ovs-tempest-slow/8773a57/testr_results.html
  - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_746/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-with-neutron-lib-master/7460037/testr_results.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2073944/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next