← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #29233

[Bug 1430751] [NEW] Launching instance that creates a new volume fails

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

I'm trying to launch instances from Horizon using the option Boot form
image - Creates a new volume.

The instance fails with block_device_mapping ERROR.

On the controller cinder/api.log and cinder/volume.log shows no error or
relevant information.

On the compute node, nova-compute.log does show the problem:

2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1819, in _prep_block_device
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     do_check_attach=do_check_attach) +
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 407, in attach_block_devices
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     map(_log_and_attach, block_device_mapping)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 405, in _log_and_attach
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     bdm.attach(*attach_args, **attach_kwargs)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 333, in attach
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     wait_func(context, vol['id'])
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1263, in _await_block_device_map_created
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     attempts=attempts)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] VolumeNotCreated: Volume abc781af-0960-4a65-87d2-a5cb15ce7273 did not finish being created even after we waited 250 seconds or 61 attempts.
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in _build_resources
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     block_device_mapping)
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1847, in _prep_block_device
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     raise exception.InvalidBDM()
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] InvalidBDM: Block Device Mapping is Invalid.
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2030, in _do_build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     filter_properties)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2129, in _build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     'create.error', fault=e)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     six.reraise(self.type_, self.value, self.tb)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2102, in _build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     block_device_mapping) as resources:
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     return self.gen.next()
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2240, in _build_resources
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     reason=msg)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] BuildAbortException: Build of instance 2da2594d-e162-45ce-98b0-54d9563bbb1c aborted: Failure prepping block device.
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 

A new volume is created, and sometimes an IP gets assigned to the
instance, other times it doesn't.

If i launch an instance choosing option Boot from image or simply boot
from volume it works, the problems seems to be when it creates a new
volume.

openstack --version
openstack 1.0.1
CentOS 7 64bits

storage mounted through nfs (wich seems to be working just fine)

Steps to reproduce the error

1 - Have storage through nfs
2 - Go to Horizon, Instances - Launch Instance -  Boot form image - Creates a new volume
3 - Choose a Windows image (or an image that big)

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: compute nova-manage

** Attachment added: "Cinder volume log"
   https://bugs.launchpad.net/bugs/1430751/+attachment/4341016/+files/cinder-volume.log

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1430751

Title:
  Launching instance that creates a new volume fails

Status in OpenStack Compute (Nova):
  New

Bug description:
  I'm trying to launch instances from Horizon using the option Boot form
  image - Creates a new volume.

  The instance fails with block_device_mapping ERROR.

  On the controller cinder/api.log and cinder/volume.log shows no error
  or relevant information.

  On the compute node, nova-compute.log does show the problem:

  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1819, in _prep_block_device
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     do_check_attach=do_check_attach) +
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 407, in attach_block_devices
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     map(_log_and_attach, block_device_mapping)
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 405, in _log_and_attach
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     bdm.attach(*attach_args, **attach_kwargs)
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 333, in attach
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     wait_func(context, vol['id'])
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1263, in _await_block_device_map_created
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     attempts=attempts)
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] VolumeNotCreated: Volume abc781af-0960-4a65-87d2-a5cb15ce7273 did not finish being created even after we waited 250 seconds or 61 attempts.
  2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in _build_resources
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     block_device_mapping)
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1847, in _prep_block_device
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     raise exception.InvalidBDM()
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] InvalidBDM: Block Device Mapping is Invalid.
  2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2030, in _do_build_and_run_instance
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     filter_properties)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2129, in _build_and_run_instance
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     'create.error', fault=e)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     six.reraise(self.type_, self.value, self.tb)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2102, in _build_and_run_instance
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     block_device_mapping) as resources:
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     return self.gen.next()
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2240, in _build_resources
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]     reason=msg)
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] BuildAbortException: Build of instance 2da2594d-e162-45ce-98b0-54d9563bbb1c aborted: Failure prepping block device.
  2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 

  A new volume is created, and sometimes an IP gets assigned to the
  instance, other times it doesn't.

  If i launch an instance choosing option Boot from image or simply boot
  from volume it works, the problems seems to be when it creates a new
  volume.

  openstack --version
  openstack 1.0.1
  CentOS 7 64bits

  storage mounted through nfs (wich seems to be working just fine)

  Steps to reproduce the error

  1 - Have storage through nfs
  2 - Go to Horizon, Instances - Launch Instance -  Boot form image - Creates a new volume
  3 - Choose a Windows image (or an image that big)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1430751/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next