yahoo-eng-team team mailing list archive

[Adam Gandelman <1200249@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova/grizzly
     Assignee: (unassigned) => Lingxian Kong (kong)

** Changed in: nova/grizzly
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1200249

Title:
  hard reboot fails when using force_raw_images=False and
  use_cow_images=False and

Status in OpenStack Compute (Nova):
  Fix Released
Status in OpenStack Compute (nova) grizzly series:
  Fix Released

Bug description:
  Repro steps:
  1) Deploy nova with force_raw_images=False and use_cow_images=False
  2) Boot an instance from a qcow2 image
  3) Wait until this instance is ACTIVE
  4) Issue a hard-reboot action on this instance

  Outcome:
  The instance transitions to HARD_REBOOT like expected, but then to SHUTOFF. The underlying VM never starts after the hard reboot action is issued.

  Expected:
  The instance should properly transition through HARD_REBOOT and back to ACTIVE. The VM should become reachable once that status transition has occurred.


  I was able to find the following log generated by nova-compute:

  2013-07-05 14:44:09.128 ERROR nova.compute.manager [req-8d0a3a55-8d5a-
  44c7-ac25-bb7dc57508ec bc7764aa97a647d19081586d69f5b444
  46ef8994dca44846ae364165c4d13357] [instance: fbc0f8c2-6743-4ddc-832c-
  1861be0b49d9] Cannot reboot instance: 'NoneType' object has no
  attribute 'rfind'' in the nova-compute log

  
  I found the source of that log statement and logged at the 'exception' level rather than the 'error level to produce this traceback:

  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1717, in reboot_instance
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] bad_volumes_callback=bad_volumes_callback)
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1303, in reboot
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] block_device_info)
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1379, in _hard_reboot
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] self._create_images_and_backing(context, instance, disk_info_json)
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3265, in _create_images_and_backing
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] cache_name = os.path.basename(info['backing_file'])
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] File "/usr/lib/python2.7/posixpath.py", line 112, in basename
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] i = p.rfind('/') + 1
  2013-07-05 14:47:02.635 13318 TRACE nova.compute.manager [instance: fbc0f8c2-6743-4ddc-832c-1861be0b49d9] AttributeError: 'NoneType' object has no attribute 'rfind'

  
  I traced back through the code and ran this command:

  $ qemu-img info /mnt/instances/fbc0f8c2-6743-4ddc-832c-1861be0b49d9/disk
  image: /mnt/instances/fbc0f8c2-6743-4ddc-832c-1861be0b49d9/disk
  file format: qcow2
  virtual size: 20G (21474836480 bytes)
  disk size: 705M
  cluster_size: 65536

  
  So it appears that Nova is expecting a backing_file key from all qcow2 images, yet when one happens  to set use_cow_images=False and force_raw_images=False that will not be true.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1200249/+subscriptions