yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1528638] [NEW] nova manage_cache task periodic task fails for ploop images

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Maxim Nestratov <mnestratov@xxxxxxxxxxxxx>
Date: Tue, 22 Dec 2015 16:48:39 -0000
Reply-to: Bug 1528638 <1528638@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

It happens because qemu-img is called for directory instead of a file in case of ploop images.
Backtrace is the following:


2015-12-17 15:46:37.917 ERROR oslo_service.periodic_task [req-1309d92e-1c47-491f-9921-93553abb3430 None None] Error during ComputeManager._run_image_cache_manager_pass
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Traceback (most recent call last):
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     task(self, context)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/compute/manager.py", line 6524, in _run_image_cache_manager_pass
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     self.driver.manage_image_cache(context, filtered_instances)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/driver.py", line 6680, in manage_image_cache
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     self.image_cache_manager.update(context, all_instances)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/imagecache.py", line 664, in update
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     self._age_and_verify_cached_images(context, all_instances, base_dir)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/imagecache.py", line 600, in _age_and_verify_cached_images
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     inuse_backing_images = self._list_backing_images()
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/imagecache.py", line 309, in _list_backing_images
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     disk_path)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/utils.py", line 181, in get_disk_backing_file
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     backing_file = images.qemu_img_info(path).backing_file
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/images.py", line 60, in qemu_img_info
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     'qemu-img', 'info', path)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/utils.py", line 391, in execute
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     return processutils.execute(*cmd, **kwargs)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 312, in execute
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     cmd=sanitized_cmd)
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task ProcessExecutionError: Unexpected error while running command.
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Command: env LC_ALL=C LANG=C qemu-img info /vzt/stack/data/nova/instances/e9bd57ca-1210-461a-b12b-1c2b9788f106/disk
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Exit code: 1
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Stdout: u''
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Stderr: u"qemu-img: Could not open '/vzt/stack/data/nova/instances/e9bd57ca-1210-461a-b12b-1c2b9788f106/disk': Could not read image for determining its format: Is a directory\n"
2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1528638

Title:
  nova manage_cache task periodic task fails for ploop images

Status in OpenStack Compute (nova):
  New

Bug description:
  It happens because qemu-img is called for directory instead of a file in case of ploop images.
  Backtrace is the following:

  
  2015-12-17 15:46:37.917 ERROR oslo_service.periodic_task [req-1309d92e-1c47-491f-9921-93553abb3430 None None] Error during ComputeManager._run_image_cache_manager_pass
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Traceback (most recent call last):
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     task(self, context)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/compute/manager.py", line 6524, in _run_image_cache_manager_pass
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     self.driver.manage_image_cache(context, filtered_instances)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/driver.py", line 6680, in manage_image_cache
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     self.image_cache_manager.update(context, all_instances)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/imagecache.py", line 664, in update
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     self._age_and_verify_cached_images(context, all_instances, base_dir)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/imagecache.py", line 600, in _age_and_verify_cached_images
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     inuse_backing_images = self._list_backing_images()
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/imagecache.py", line 309, in _list_backing_images
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     disk_path)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/libvirt/utils.py", line 181, in get_disk_backing_file
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     backing_file = images.qemu_img_info(path).backing_file
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/virt/images.py", line 60, in qemu_img_info
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     'qemu-img', 'info', path)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/vzt/stack/nova/nova/utils.py", line 391, in execute
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     return processutils.execute(*cmd, **kwargs)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task   File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 312, in execute
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task     cmd=sanitized_cmd)
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task ProcessExecutionError: Unexpected error while running command.
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Command: env LC_ALL=C LANG=C qemu-img info /vzt/stack/data/nova/instances/e9bd57ca-1210-461a-b12b-1c2b9788f106/disk
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Exit code: 1
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Stdout: u''
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task Stderr: u"qemu-img: Could not open '/vzt/stack/data/nova/instances/e9bd57ca-1210-461a-b12b-1c2b9788f106/disk': Could not read image for determining its format: Is a directory\n"
  2015-12-17 15:46:37.917 23014 ERROR oslo_service.periodic_task

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1528638/+subscriptions
Follow ups

[Bug 1528638] Fix merged to nova (master)
From: OpenStack Infra, 2016-06-27