yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1431260] Re: Instance failed block device setup

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: John Griffith <1431260@xxxxxxxxxxxxxxxxxx>
Date: Fri, 27 Mar 2015 20:06:48 -0000
Reply-to: Bug 1431260 <1431260@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
so this is typically because on some backends the image-
download/conversion can take a relatively long time.  The feature of
rolling this up into one command in Horizon was a good idea, but it
unfortunately doesn't coordinate things very well or check status before
trying to move on.

this is actually an issue in Horizon as there's not direct API call that
does this but instead it's a chained sequence built into Horizon.

Work around as you've already figured out is do things in two steps 1.
Create bootable volume, 2. Boot the volume.

** Project changed: cinder => horizon

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1431260

Title:
  Instance failed block device setup

Status in OpenStack Dashboard (Horizon):
  New

Bug description:
  I'm trying to launch instances from Horizon using the option Boot form
  image - Creates a new volume.

  The instance fails with block_device_mapping ERROR.

  On the controller cinder/api.log and cinder/volume.log shows no error
  or relevant information.

  On the compute node, nova-compute.log does show the problem:

  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1819, in _prep_block_device
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] do_check_attach=do_check_attach) +
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 407, in attach_block_devices
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] map(_log_and_attach, block_device_mapping)
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 405, in _log_and_attach
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] bdm.attach(*attach_args, **attach_kwargs)
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 333, in attach
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] wait_func(context, vol['id'])
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1263, in _await_block_device_map_created
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] attempts=attempts)
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] VolumeNotCreated: Volume abc781af-0960-4a65-87d2-a5cb15ce7273 did not finish being created even after we waited 250 seconds or 61 attempts.
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in _build_resources
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] block_device_mapping)
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1847, in _prep_block_device
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] raise exception.InvalidBDM()
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] InvalidBDM: Block Device Mapping is Invalid.
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2030, in _do_build_and_run_instance
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] filter_properties)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2129, in _build_and_run_instance
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 'create.error', fault=e)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] six.reraise(self.type_, self.value, self.tb)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2102, in _build_and_run_instance
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] block_device_mapping) as resources:
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] return self.gen.next()
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2240, in _build_resources
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] reason=msg)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] BuildAbortException: Build of instance 2da2594d-e162-45ce-98b0-54d9563bbb1c aborted: Failure prepping block device.
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]

  A new volume is created, and sometimes an IP gets assigned to the
  instance, other times it doesn't.

  If i launch an instance choosing option Boot from image or simply boot
  from volume it works, the problems seems to be when it creates a new
  volume.

  openstack --version
  openstack 1.0.1
  CentOS 7 64bits

  storage mounted through nfs (wich seems to be working just fine)

  Steps to reproduce the error

  1 - Have storage through nfs
  2 - Go to Horizon, Instances - Launch Instance - Boot form image - Creates a new volume
  3 - Choose a Windows image (or an image that big)

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1431260/+subscriptions