← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #84960

[Bug 1912607] [NEW] test_attach_cloned_encrypted_volume fails in nova-ceph-multistore job permanently

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

The job is red since 2021.01.20 20:18 UTC[1]. The failure is:

tempest.exceptions.BuildErrorException: Server 79ce2743-1b98-4867-8396-5c68c25f2cf3 failed to build and is in ERROR status
Details: {'code': 500, 'created': '2021-01-20T22:28:59Z', 'message': 'Build of instance 79ce2743-1b98-4867-8396-5c68c25f2cf3 aborted: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format'}


Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [None req-3a8ebd8e-b22a-446f-ad35-16b12e7a1182 tempest-TestEncryptedCinderVolumes-88193216 tempest-TestEncryptedCinderVolumes-88193216] [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] Instance failed to spawn: nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3] Traceback (most recent call last):

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/compute/manager.py", line 2615, in
_build_resources

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     yield resources

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/compute/manager.py", line 2384, in
_build_and_run_instance

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     self.driver.spawn(context,
instance, image_meta,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 4061, in spawn

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     created_instance_dir,
created_disks = self._create_image(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 4457, in
_create_image

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     created_disks =
self._create_and_inject_local_root(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 4585, in
_create_and_inject_local_root

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]
self._try_fetch_image_cache(backend, fetch_func, context,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 10164, in
_try_fetch_image_cache

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]
image.cache(fetch_func=fetch_func,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 273, in cache

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]
self.create_image(fetch_func_sync, base, size,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 939, in
create_image

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     prepare_template(target=base,
*args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
/dist-packages/oslo_concurrency/lockutils.py", line 360, in inner

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     return f(*args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 270, in
fetch_func_sync

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     fetch_func(target=target,
*args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 4572, in
clone_fallback_to_fetch

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     LOG.warning(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
/dist-packages/oslo_utils/excutils.py", line 220, in __exit__

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     self.force_reraise()

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     six.reraise(self.type_,
self.value, self.tb)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
/dist-packages/six.py", line 703, in reraise

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     raise value

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/driver.py", line 4564, in
clone_fallback_to_fetch

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     backend.clone(context,
disk_images['image_id'])

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
"/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 1039, in clone

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]     raise
exception.ImageUnacceptable(image_id=image_id_or_uri,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3] nova.exception.ImageUnacceptable:
Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not
raw format

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
compute[94729]: ERROR nova.compute.manager [instance:
79ce2743-1b98-4867-8396-5c68c25f2cf3]

Logstash signature:
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22is%20unacceptable%3A%20Image%20is%20not%20raw%20format'%5C%22


[1] https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack/nova

** Affects: nova
     Importance: Critical
         Status: New


** Tags: gate-failure

** Changed in: nova
   Importance: Undecided => Critical

** Tags added: gate-failure

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1912607

Title:
  test_attach_cloned_encrypted_volume fails in nova-ceph-multistore job
  permanently

Status in OpenStack Compute (nova):
  New

Bug description:
  The job is red since 2021.01.20 20:18 UTC[1]. The failure is:

  tempest.exceptions.BuildErrorException: Server 79ce2743-1b98-4867-8396-5c68c25f2cf3 failed to build and is in ERROR status
  Details: {'code': 500, 'created': '2021-01-20T22:28:59Z', 'message': 'Build of instance 79ce2743-1b98-4867-8396-5c68c25f2cf3 aborted: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format'}

  
  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [None req-3a8ebd8e-b22a-446f-ad35-16b12e7a1182 tempest-TestEncryptedCinderVolumes-88193216 tempest-TestEncryptedCinderVolumes-88193216] [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] Instance failed to spawn: nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3] Traceback (most recent call
  last):

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/compute/manager.py", line 2615, in
  _build_resources

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     yield resources

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/compute/manager.py", line 2384, in
  _build_and_run_instance

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     self.driver.spawn(context,
  instance, image_meta,

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4061, in spawn

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     created_instance_dir,
  created_disks = self._create_image(

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4457, in
  _create_image

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     created_disks =
  self._create_and_inject_local_root(

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4585, in
  _create_and_inject_local_root

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]
  self._try_fetch_image_cache(backend, fetch_func, context,

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/driver.py", line 10164, in
  _try_fetch_image_cache

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]
  image.cache(fetch_func=fetch_func,

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 273, in
  cache

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]
  self.create_image(fetch_func_sync, base, size,

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 939, in
  create_image

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]
  prepare_template(target=base, *args, **kwargs)

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
  /dist-packages/oslo_concurrency/lockutils.py", line 360, in inner

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     return f(*args, **kwargs)

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 270, in
  fetch_func_sync

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     fetch_func(target=target,
  *args, **kwargs)

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4572, in
  clone_fallback_to_fetch

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     LOG.warning(

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
  /dist-packages/oslo_utils/excutils.py", line 220, in __exit__

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     self.force_reraise()

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
  /dist-packages/oslo_utils/excutils.py", line 196, in force_reraise

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     six.reraise(self.type_,
  self.value, self.tb)

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File "/usr/local/lib/python3.8
  /dist-packages/six.py", line 703, in reraise

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     raise value

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4564, in
  clone_fallback_to_fetch

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     backend.clone(context,
  disk_images['image_id'])

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]   File
  "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 1039, in
  clone

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]     raise
  exception.ImageUnacceptable(image_id=image_id_or_uri,

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]
  nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-
  9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

  Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-
  compute[94729]: ERROR nova.compute.manager [instance:
  79ce2743-1b98-4867-8396-5c68c25f2cf3]

  Logstash signature:
  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22is%20unacceptable%3A%20Image%20is%20not%20raw%20format'%5C%22

  
  [1] https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack/nova

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1912607/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next