yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #29233
[Bug 1430751] [NEW] Launching instance that creates a new volume fails
Public bug reported:
I'm trying to launch instances from Horizon using the option Boot form
image - Creates a new volume.
The instance fails with block_device_mapping ERROR.
On the controller cinder/api.log and cinder/volume.log shows no error or
relevant information.
On the compute node, nova-compute.log does show the problem:
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1819, in _prep_block_device
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] do_check_attach=do_check_attach) +
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 407, in attach_block_devices
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] map(_log_and_attach, block_device_mapping)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 405, in _log_and_attach
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] bdm.attach(*attach_args, **attach_kwargs)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 333, in attach
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] wait_func(context, vol['id'])
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1263, in _await_block_device_map_created
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] attempts=attempts)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] VolumeNotCreated: Volume abc781af-0960-4a65-87d2-a5cb15ce7273 did not finish being created even after we waited 250 seconds or 61 attempts.
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in _build_resources
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] block_device_mapping)
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1847, in _prep_block_device
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] raise exception.InvalidBDM()
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] InvalidBDM: Block Device Mapping is Invalid.
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2030, in _do_build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] filter_properties)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2129, in _build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 'create.error', fault=e)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] six.reraise(self.type_, self.value, self.tb)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2102, in _build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] block_device_mapping) as resources:
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] return self.gen.next()
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2240, in _build_resources
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] reason=msg)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] BuildAbortException: Build of instance 2da2594d-e162-45ce-98b0-54d9563bbb1c aborted: Failure prepping block device.
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
A new volume is created, and sometimes an IP gets assigned to the
instance, other times it doesn't.
If i launch an instance choosing option Boot from image or simply boot
from volume it works, the problems seems to be when it creates a new
volume.
openstack --version
openstack 1.0.1
CentOS 7 64bits
storage mounted through nfs (wich seems to be working just fine)
Steps to reproduce the error
1 - Have storage through nfs
2 - Go to Horizon, Instances - Launch Instance - Boot form image - Creates a new volume
3 - Choose a Windows image (or an image that big)
** Affects: nova
Importance: Undecided
Status: New
** Tags: compute nova-manage
** Attachment added: "Cinder volume log"
https://bugs.launchpad.net/bugs/1430751/+attachment/4341016/+files/cinder-volume.log
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1430751
Title:
Launching instance that creates a new volume fails
Status in OpenStack Compute (Nova):
New
Bug description:
I'm trying to launch instances from Horizon using the option Boot form
image - Creates a new volume.
The instance fails with block_device_mapping ERROR.
On the controller cinder/api.log and cinder/volume.log shows no error
or relevant information.
On the compute node, nova-compute.log does show the problem:
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1819, in _prep_block_device
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] do_check_attach=do_check_attach) +
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 407, in attach_block_devices
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] map(_log_and_attach, block_device_mapping)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 405, in _log_and_attach
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] bdm.attach(*attach_args, **attach_kwargs)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 333, in attach
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] wait_func(context, vol['id'])
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1263, in _await_block_device_map_created
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] attempts=attempts)
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] VolumeNotCreated: Volume abc781af-0960-4a65-87d2-a5cb15ce7273 did not finish being created even after we waited 250 seconds or 61 attempts.
2015-03-11 11:23:02.807 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in _build_resources
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] block_device_mapping)
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1847, in _prep_block_device
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] raise exception.InvalidBDM()
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] InvalidBDM: Block Device Mapping is Invalid.
2015-03-11 11:23:02.809 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] Traceback (most recent call last):
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2030, in _do_build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] filter_properties)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2129, in _build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] 'create.error', fault=e)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] six.reraise(self.type_, self.value, self.tb)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2102, in _build_and_run_instance
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] block_device_mapping) as resources:
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] return self.gen.next()
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2240, in _build_resources
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] reason=msg)
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c] BuildAbortException: Build of instance 2da2594d-e162-45ce-98b0-54d9563bbb1c aborted: Failure prepping block device.
2015-03-11 11:23:02.848 72172 TRACE nova.compute.manager [instance: 2da2594d-e162-45ce-98b0-54d9563bbb1c]
A new volume is created, and sometimes an IP gets assigned to the
instance, other times it doesn't.
If i launch an instance choosing option Boot from image or simply boot
from volume it works, the problems seems to be when it creates a new
volume.
openstack --version
openstack 1.0.1
CentOS 7 64bits
storage mounted through nfs (wich seems to be working just fine)
Steps to reproduce the error
1 - Have storage through nfs
2 - Go to Horizon, Instances - Launch Instance - Boot form image - Creates a new volume
3 - Choose a Windows image (or an image that big)
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1430751/+subscriptions
Follow ups
References