yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1253508] Re: Instances fail to spawn, e2label error when using raw

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sam Morrison <1253508@xxxxxxxxxxxxxxxxxx>
Date: Thu, 21 Nov 2013 03:56:32 -0000
Reply-to: Bug 1253508 <1253508@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Actually, I've just realised this is a simple spelling mistake, created
a new bug and will mark this invalid.

See bug 1253510


** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1253508

Title:
  Instances fail to spawn, e2label error when using raw

Status in OpenStack Compute (Nova):
  Invalid

Bug description:
  We've just upgraded our test nova environment from grizzly -> havana
  and we are getting the following error when booting an instance.

  This error happens when setting:

  force_raw_images = True
  use_cow_images = False

  We are using the latest checkout from stable/havana (HEAD =
  f7cfc1cc6ba51617d02dd2b55419fad52a51793b)

  2013-11-21 14:42:17.225 1250 DEBUG nova.virt.disk.api [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Checking if we can resize image /var/lib/nova/instances/bbda2f1d-e682-4300-b9a2-fde84e2e0453/disk. size=21474836480 can_resize_image /usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:157
  2013-11-21 14:42:17.227 1250 DEBUG nova.openstack.common.processutils [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/bbda2f1d-e682-4300-b9a2-fde84e2e0453/disk execute /usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py:147
  2013-11-21 14:42:17.578 1250 DEBUG nova.openstack.common.processutils [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Running cmd (subprocess): qemu-img resize /var/lib/nova/instances/bbda2f1d-e682-4300-b9a2-fde84e2e0453/disk 21474836480 execute /usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py:147
  2013-11-21 14:42:17.792 1250 DEBUG nova.virt.disk.api [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Checking if we can resize filesystem inside /var/lib/nova/instances/bbda2f1d-e682-4300-b9a2-fde84e2e0453/disk. CoW=False is_image_partitionless /usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:171
  2013-11-21 14:42:17.794 1250 DEBUG nova.openstack.common.processutils [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Running cmd (subprocess): e2label /var/lib/nova/instances/bbda2f1d-e682-4300-b9a2-fde84e2e0453/disk execute /usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py:147
  2013-11-21 14:42:17.813 1250 DEBUG nova.openstack.common.processutils [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Result was 1 execute /usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py:172
  2013-11-21 14:42:17.817 1250 DEBUG nova.openstack.common.lockutils [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] Released file lock "/var/lib/nova/instances/_base/5420f3caf82c8bd8dfeb541c03ed33f1a1631ea5" at /var/lib/nova/instances/locks/nova-_var_lib_nova_instances__base_5420f3caf82c8bd8dfeb541c03ed33f1a1631ea5 lock /usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py:208
  2013-11-21 14:42:19.702 1250 ERROR nova.compute.manager [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453] Instance failed to spawn
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453] Traceback (most recent call last):
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1407, in _spawn
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     block_device_info)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2063, in spawn
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     admin_pass=admin_password)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2353, in _create_image
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     project_id=instance['project_id'])
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 174, in cache
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     *args, **kwargs)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 271, in create_image
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     copy_raw_image(base, self.path, size)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line 246, in inner
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     return f(*args, **kwargs)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 260, in copy_raw_image
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     disk.extend(target, size, use_cow=use_cow)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py", line 137, in extend
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     if not is_image_partitionless(image, use_cow):
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py", line 193, in is_image_partitionless
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     'error': e})
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 1412, in debug
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     self.logger.debug(msg, *args, **kwargs)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 1128, in debug
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     self._log(DEBUG, msg, args, **kwargs)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 1258, in _log
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     self.handle(record)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 1268, in handle
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     self.callHandlers(record)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 1308, in callHandlers
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     hdlr.handle(record)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 748, in handle
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     self.emit(record)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/handlers.py", line 791, in emit
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     msg = self.format(record) + '\000'
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     return fmt.format(record)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/log.py", line 517, in format
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     return logging.Formatter.format(self, record)
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     record.message = record.getMessage()
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]   File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]     msg = msg % self.args
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453] KeyError: u'errror'
  2013-11-21 14:42:19.702 1250 TRACE nova.compute.manager [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453]
  2013-11-21 14:42:19.710 1250 DEBUG nova.compute.claims [req-1b530b18-b502-4319-b099-7d7853f79fab d9646718471b46aeb5fd94c702336ca9 0bdf024c921848c4b74d9e69af9edf08] [instance: bbda2f1d-e682-4300-b9a2-fde84e2e0453] Aborting claim: [Claim: 2048 MB memory, 20 GB disk, 1 VCPUS] abort /usr/lib/python2.7/dist-packages/nova/compute/claims.py:107

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1253508/+subscriptions