← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #35430

[Bug 1318544] Re: XenServer - Nova-Compute StorageError Waiting for device

  Thread Previous • Date Previous • • Thread Next 
** Description changed:

  Hi All,
  
  I started building a Openstack cloud based on the new LTS version of
  Ubuntu (14.04). I installed both Control and Compute nodes as VM's on a
  XenServer. I selected the option 'Other Operating System' so this is
  running as HVM.
  
  The cluster is up and running: i can start instances, create storage and
  allocate IP addresses. But the error i got is popping up when the
  instance is in spawning state. The error i attached is from the nova-
  compute node.
  
- I use a IBM storwize backend with Cinder connection using iSCSI. I can
- see the instances being created, the storage being connected but the
- following error is keep coming back.
+ ...
  
- Locally on the XenServer SDB and SDC existed, which leaded to a earlier
- error in which HDB was HDC. I removed SDC locally from the XenServer
- using: "echo 1 > /sys/block/sdc/device/delete". This changed the HDC
- error message to HDB. The same thing i did for SDB. Both where virtual
- drives from DRAC. But still this message is keep popping up with HDB.
- 
- I read an other article about a similar error, this guy was pointed in
- the direction of a HVM vs PV solution because the VM is communicating
- with the DOM0 via it's kernel. I checked and after installing the
- XenServer tools the UUID is there and is readable.
- 
- While trying the solution above i also focused on the HDB reference,
- this was strange because with all the rewrite options in nova i would
- expect a reference to xvda: "xenapi_remap_vbd_dev=false"
- 
- Could you guys point me in the right direction or help me solve this
- problem? I think this isn't a mis-configuration because i tried all
- possible configurations and this message keeps popping up.
- 
-  This is the last step towards a working openstack cloud with Ubuntu
- 14.04 and XenServer 6.2 all patches applied.
- 
- 2014-05-12 09:00:30.918 6660 ERROR nova.compute.manager [req-595bc50b-e8b2-4de4-8c38-09887c8d4c82 f8609ec1f5254fcfb1fdf6df76876805 41971b840c7a405d9da5052a2506a1c9] [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9] Error: Timeout waiting for device hdb to be created
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9] Traceback (most recent call last):
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1311, in _build_instance
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     set_access_ip=set_access_ip)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 399, in decorated_function
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     return function(self, context, *args, **kwargs)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1723, in _spawn
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     LOG.exception(_('Instance failed to spawn'), instance=instance)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     six.reraise(self.type_, self.value, self.tb)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1720, in _spawn
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     block_device_info)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/driver.py", line 230, in spawn
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     admin_password, network_info, block_device_info)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 357, in spawn
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     network_info, block_device_info, name_label, rescue)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 526, in _spawn
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     undo_mgr.rollback_and_reraise(msg=msg, instance=instance)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 830, in rollback_and_reraise
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     self._rollback()
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     six.reraise(self.type_, self.value, self.tb)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 509, in _spawn
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     attach_devices_step(undo_mgr, vm_ref, vdis, disk_image_type)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 146, in inner
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     rv = f(*args, **kwargs)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 443, in attach_devices_step
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     attach_disks(undo_mgr, vm_ref, vdis, disk_image_type)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 390, in attach_disks
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     injected_files)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 595, in _attach_disks
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     flavor['root_gb'])
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vm_utils.py", line 1037, in try_auto_configure_disk
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     _auto_configure_disk(session, vdi_ref, new_gb)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vm_utils.py", line 1018, in _auto_configure_disk
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     with vdi_attached_here(session, vdi_ref, read_only=False) as dev:
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     return self.gen.next()
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vm_utils.py", line 2200, in vdi_attached_here
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     _wait_for_device(dev)
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]   File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vm_utils.py", line 2155, in _wait_for_device
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]     _('Timeout waiting for device %s to be created') % dev)
  2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9] StorageError: Timeout waiting for device hdb to be created
- 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9] 
- 
- Best,
- - Thijs
+ 2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]

** Changed in: nova
       Status: Invalid => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1318544

Title:
  XenServer - Nova-Compute StorageError Waiting for device

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  Hi All,

  I started building a Openstack cloud based on the new LTS version of
  Ubuntu (14.04). I installed both Control and Compute nodes as VM's on
  a XenServer. I selected the option 'Other Operating System' so this is
  running as HVM.

  The cluster is up and running: i can start instances, create storage
  and allocate IP addresses. But the error i got is popping up when the
  instance is in spawning state. The error i attached is from the nova-
  compute node.

  ...

  2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9] StorageError: Timeout waiting for device hdb to be created
  2014-05-12 09:00:30.918 6660 TRACE nova.compute.manager [instance: 00e2c60e-372b-43d2-a2fd-bcc3628c24a9]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1318544/+subscriptions

References

  Thread Previous • Date Previous • • Thread Next