← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #22364

[Bug 1373962] [NEW] LVM backed VM fails to launch

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

LVM ephemeral storage backend is broken in the most recent Nova (commit
945646e1298a53be6ae284766f5023d754dfe57d)

To reproduce in Devstack:

1. Configure Nova to use LVM ephemeral storage by adding to
create_nova_conf function in lib/nova

        iniset $NOVA_CONF libvirt images_type "lvm"
        iniset $NOVA_CONF libvirt images_volume_group "nova-lvm"

2. Create a backing file for LVM

        truncate -s 5G nova-backing-file

3. Mount the file via loop device

        sudo losetup /dev/loop0 nova-backing-file

4. Create nova-lvm volume group

        sudo vgcreate nova-lvm /dev/loop0

5. Launch Devstack

6. Alternatively, skipping step 1, /etc/nova/nova.conf can be modified
after Devstack is launched by adding

        [libvirt]
        images_type = lvm
        images_volume_group = nova-lvm

    and then restarting nova-compute by entering the Devstack screen
session, going to the n-cpu screen and hitting Ctrl-C, Up-arrow, and
Enter.

7. Launch an instance

        nova boot test --flavor 1 --image cirros-0.3.2-x86_64-uec

Instance fails to launch. Nova compute reports

2014-09-25 10:11:08.180 ERROR nova.compute.manager [req-b7924ad0-5f4b-46eb-a798-571d97c77145 demo demo] [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Instance failed to spawn
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Traceback (most recent call last):
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/compute/manager.py", line 2231, in _build_resources
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     yield resources
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/compute/manager.py", line 2101, in _build_and_run_instance
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     block_device_info=block_device_info)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2617, in spawn
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     block_device_info, disk_info=disk_info)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4434, in _create_domain_and_network
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     domain.destroy()
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4358, in _create_domain
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     for vif in network_info if vif.get('active', True) is False]
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 82, in __exit__
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     six.reraise(self.type_, self.value, self.tb)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4349, in _create_domain
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     raise exception.VirtualInterfaceCreateException()
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     rv = execute(f, *args, **kwargs)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     six.reraise(c, e, tb)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     rv = meth(*args, **kwargs)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 711, in createWithFlags
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] libvirtError: cannot open file '/dev/nova-lvm/8d69f0de-253a-403e-a137-0da77b0d415c_disk.config': No such file or directory
2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]
2014-09-25 10:11:08.218 AUDIT nova.compute.manager [req-b7924ad0-5f4b-46eb-a798-571d97c77145 demo demo] [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Terminating instance
2014-09-25 10:11:08.228 29649 INFO nova.virt.libvirt.driver [-] [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Instance destroyed successfully.

** Affects: nova
     Importance: High
         Status: New


** Tags: backed instance lvm nova

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1373962

Title:
  LVM backed VM fails to launch

Status in OpenStack Compute (Nova):
  New

Bug description:
  LVM ephemeral storage backend is broken in the most recent Nova
  (commit 945646e1298a53be6ae284766f5023d754dfe57d)

  To reproduce in Devstack:

  1. Configure Nova to use LVM ephemeral storage by adding to
  create_nova_conf function in lib/nova

          iniset $NOVA_CONF libvirt images_type "lvm"
          iniset $NOVA_CONF libvirt images_volume_group "nova-lvm"

  2. Create a backing file for LVM

          truncate -s 5G nova-backing-file

  3. Mount the file via loop device

          sudo losetup /dev/loop0 nova-backing-file

  4. Create nova-lvm volume group

          sudo vgcreate nova-lvm /dev/loop0

  5. Launch Devstack

  6. Alternatively, skipping step 1, /etc/nova/nova.conf can be modified
  after Devstack is launched by adding

          [libvirt]
          images_type = lvm
          images_volume_group = nova-lvm

      and then restarting nova-compute by entering the Devstack screen
  session, going to the n-cpu screen and hitting Ctrl-C, Up-arrow, and
  Enter.

  7. Launch an instance

          nova boot test --flavor 1 --image cirros-0.3.2-x86_64-uec

  Instance fails to launch. Nova compute reports

  2014-09-25 10:11:08.180 ERROR nova.compute.manager [req-b7924ad0-5f4b-46eb-a798-571d97c77145 demo demo] [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Instance failed to spawn
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Traceback (most recent call last):
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/compute/manager.py", line 2231, in _build_resources
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     yield resources
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/compute/manager.py", line 2101, in _build_and_run_instance
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     block_device_info=block_device_info)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2617, in spawn
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     block_device_info, disk_info=disk_info)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4434, in _create_domain_and_network
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     domain.destroy()
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4358, in _create_domain
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     for vif in network_info if vif.get('active', True) is False]
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/openstack/common/excutils.py", line 82, in __exit__
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     six.reraise(self.type_, self.value, self.tb)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4349, in _create_domain
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     raise exception.VirtualInterfaceCreateException()
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     rv = execute(f, *args, **kwargs)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     six.reraise(c, e, tb)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     rv = meth(*args, **kwargs)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 711, in createWithFlags
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] libvirtError: cannot open file '/dev/nova-lvm/8d69f0de-253a-403e-a137-0da77b0d415c_disk.config': No such file or directory
  2014-09-25 10:11:08.180 29649 TRACE nova.compute.manager [instance: 8d69f0de-253a-403e-a137-0da77b0d415c]
  2014-09-25 10:11:08.218 AUDIT nova.compute.manager [req-b7924ad0-5f4b-46eb-a798-571d97c77145 demo demo] [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Terminating instance
  2014-09-25 10:11:08.228 29649 INFO nova.virt.libvirt.driver [-] [instance: 8d69f0de-253a-403e-a137-0da77b0d415c] Instance destroyed successfully.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1373962/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next