← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #08497

[Bug 1271562] [NEW] Failed deployment of VM due to the use of guestfs.GuestFS (close_on_exit = False)

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

After Bug 1261475 was fixed, A new issue in Ice-house was introduced.
Basically now that ice-house is doing the call guestfs.GuestFS
(close_on_exit = False), there are different requirements on the version
of libguestfs.x86_64 and python-libguestfs.x86_64.

During the deployment of a VM I would get the following error on the
compute node:

2014-01-20 16:51:32.395 7723 ERROR nova.compute.manager [req-482a2eec-238c-4696-8aab-acc0d404e627 45a71543cb6f4022ac4f56855a7ff197 4a5f610e660347b6a629a27ffea92240] [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] Error: __init__() got an unexpected keyword argument 'close_on_exit'
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] Traceback (most recent call last):
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1062, in _build_instance
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] set_access_ip=set_access_ip)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 362, in decorated_function
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] return function(self, context, *args, **kwargs)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1497, in _spawn
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] LOG.exception(_('Instance failed to spawn'), instance=instance)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] six.reraise(self.type_, self.value, self.tb)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1483, in _spawn
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] block_device_info)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2174, in spawn
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] admin_pass=admin_password)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2477, in _create_image
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] project_id=instance['project_id'])
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 187, in cache
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] *args, **kwargs)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 337, in create_image
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] copy_qcow2_image(base, self.path, size)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] return f(*args, **kwargs)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 303, in copy_qcow2_image
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] disk.extend(target, size, use_cow=True)
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/disk/api.py", line 149, in extend
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] if not is_image_partitionless(image, use_cow):
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/disk/api.py", line 189, in is_image_partitionless
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] fs.setup()
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/disk/vfs/guestfs.py", line 111, in setup
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] self.handle = tpool.Proxy(guestfs.GuestFS(close_on_exit=False))
2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] TypeError: __init__() got an unexpected keyword argument 'close_on_exit'

After doing some research I found out that the close_on_exit parameter
is supported only on the newer versions of libguestfs.x86_64 and python-
libguestfs.x86_64.

In my system (RHEL 6.4) I had
libguestfs.x86_64 1:1.16.34-2.el6
python-libguestfs.x86_64 1:1.16.34-2.el6

After updating to the following versions the deploy operation started working again.
libguestfs.x86_64 1:1.20.11-2.el6
python-libguestfs.x86_64 1:1.20.11-2.el6

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: compute

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1271562

Title:
  Failed deployment of VM due to the use of guestfs.GuestFS
  (close_on_exit = False)

Status in OpenStack Compute (Nova):
  New

Bug description:
  After Bug 1261475 was fixed, A new issue in Ice-house was introduced.
  Basically now that ice-house is doing the call guestfs.GuestFS
  (close_on_exit = False), there are different requirements on the
  version of libguestfs.x86_64 and python-libguestfs.x86_64.

  During the deployment of a VM I would get the following error on the
  compute node:

  2014-01-20 16:51:32.395 7723 ERROR nova.compute.manager [req-482a2eec-238c-4696-8aab-acc0d404e627 45a71543cb6f4022ac4f56855a7ff197 4a5f610e660347b6a629a27ffea92240] [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] Error: __init__() got an unexpected keyword argument 'close_on_exit'
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] Traceback (most recent call last):
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1062, in _build_instance
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] set_access_ip=set_access_ip)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 362, in decorated_function
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] return function(self, context, *args, **kwargs)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1497, in _spawn
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] LOG.exception(_('Instance failed to spawn'), instance=instance)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] six.reraise(self.type_, self.value, self.tb)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1483, in _spawn
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] block_device_info)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2174, in spawn
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] admin_pass=admin_password)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 2477, in _create_image
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] project_id=instance['project_id'])
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 187, in cache
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] *args, **kwargs)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 337, in create_image
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] copy_qcow2_image(base, self.path, size)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/openstack/common/lockutils.py", line 249, in inner
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] return f(*args, **kwargs)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/imagebackend.py", line 303, in copy_qcow2_image
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] disk.extend(target, size, use_cow=True)
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/disk/api.py", line 149, in extend
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] if not is_image_partitionless(image, use_cow):
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/disk/api.py", line 189, in is_image_partitionless
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] fs.setup()
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] File "/usr/lib/python2.6/site-packages/nova/virt/disk/vfs/guestfs.py", line 111, in setup
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] self.handle = tpool.Proxy(guestfs.GuestFS(close_on_exit=False))
  2014-01-20 16:51:32.395 7723 TRACE nova.compute.manager [instance: d4083a7f-0c2d-44b3-a79b-7f498ea75ecd] TypeError: __init__() got an unexpected keyword argument 'close_on_exit'

  After doing some research I found out that the close_on_exit parameter
  is supported only on the newer versions of libguestfs.x86_64 and
  python-libguestfs.x86_64.

  In my system (RHEL 6.4) I had
  libguestfs.x86_64 1:1.16.34-2.el6
  python-libguestfs.x86_64 1:1.16.34-2.el6

  After updating to the following versions the deploy operation started working again.
  libguestfs.x86_64 1:1.20.11-2.el6
  python-libguestfs.x86_64 1:1.20.11-2.el6

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1271562/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next