yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1436187] Re: 'AttributeError' is getting raised while unshelving instance booted from volume

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Dave Walker <email@xxxxxxxxxx>
Date: Thu, 21 Jan 2016 20:29:59 -0000
Reply-to: Bug 1436187 <1436187@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova/kilo
       Status: Fix Released => Fix Committed

** Changed in: nova/kilo
    Milestone: 2015.1.2 => 2015.1.3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1436187

Title:
  'AttributeError' is getting raised while unshelving instance booted
  from volume

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) kilo series:
  Fix Committed

Bug description:
  ‘AttributeError’ exception is getting raised while unshelving instance
  which is booted from volume

  Steps to reproduce:
  ------------------------
  1.	Create bootable volume
  2.	Create instance from bootable volume
  3.	Shelve instance
  4.	Try to unshelve instance

  Error log on n-cpu service:
  ---------------------------------------

  2015-03-24 23:32:13.646 ERROR nova.compute.manager [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a] Instance failed to spawn
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a] Traceback (most recent call last):
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]   File "/opt/stack/nova/nova/compute/manager.py", line 4368, in _unshelve_instance
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]     block_device_info=block_device_info)
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2342, in spawn
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]     block_device_info)
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]   File "/opt/stack/nova/nova/virt/libvirt/blockinfo.py", line 622, in get_disk_info
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]     instance=instance)
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]   File "/opt/stack/nova/nova/virt/libvirt/blockinfo.py", line 232, in get_disk_bus_for_device_type
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]     disk_bus = image_meta.get('properties', {}).get(key)
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a] AttributeError: 'NoneType' object has no attribute 'get'
  2015-03-24 23:32:13.646 TRACE nova.compute.manager [instance: 183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a]
  2015-03-24 23:32:13.649 DEBUG oslo_concurrency.lockutils [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] Lock "183c8ba3-3074-47aa-bc0c-bd9cea9c6d2a" released by "do_unshelve_instance" :: held 1.182s from (pid=11271) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
  2015-03-24 23:32:13.650 DEBUG oslo_messaging._drivers.amqpdriver [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] MSG_ID is 9c227430eaf34c64b94f36661ef2ec8f from (pid=11271) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:310
  2015-03-24 23:32:13.650 DEBUG oslo_messaging._drivers.amqp [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] UNIQUE_ID is 7329362a2cab48968ce31760bcac8628. from (pid=11271) _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:226
  2015-03-24 23:32:13.696 DEBUG oslo_messaging._drivers.amqpdriver [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] MSG_ID is d2388c787036413a9bcf95f55e38027b from (pid=11271) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:310
  2015-03-24 23:32:13.696 DEBUG oslo_messaging._drivers.amqp [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] UNIQUE_ID is c466ff4a11574ff3a1032e85f3d9bd87. from (pid=11271) _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:226
  2015-03-24 23:32:13.746 DEBUG oslo_messaging._drivers.amqpdriver [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] MSG_ID is 0367b08fd7dd428ab8ef494bb42f1499 from (pid=11271) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:310
  2015-03-24 23:32:13.746 DEBUG oslo_messaging._drivers.amqp [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] UNIQUE_ID is b7395e5e66da4a47ba4132568713d4c4. from (pid=11271) _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:226
  2015-03-24 23:32:13.789 DEBUG nova.openstack.common.periodic_task [req-db2bb34f-1f3d-4ac0-99d0-f6fe78f8393d None None] Running periodic task ComputeManager._poll_volume_usage from (pid=11271) run_periodic_tasks /opt/stack/nova/nova/openstack/common/periodic_task.py:219
  2015-03-24 23:32:13.789 DEBUG nova.openstack.common.loopingcall [req-db2bb34f-1f3d-4ac0-99d0-f6fe78f8393d None None] Dynamic looping call <bound method Service.periodic_tasks of <nova.service.Service object at 0x7f3fad97c450>> sleeping for 3.00 seconds from (pid=11271) _inner /opt/stack/nova/nova/openstack/common/loopingcall.py:132
  2015-03-24 23:32:13.845 DEBUG oslo_concurrency.lockutils [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] Lock "compute_resources" acquired by "update_usage" :: waited 0.000s from (pid=11271) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444
  2015-03-24 23:32:13.845 DEBUG oslo_concurrency.lockutils [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] Lock "compute_resources" released by "update_usage" :: held 0.000s from (pid=11271) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456
  2015-03-24 23:32:13.848 ERROR oslo_messaging.rpc.dispatcher [req-da05280c-be61-4705-a0c9-08ccf3c1d245 demo demo] Exception during message handling: 'NoneType' object has no attribute 'get'
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 88, in wrapped
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     payload)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/exception.py", line 71, in wrapped
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 310, in decorated_function
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance_uuid=instance_uuid)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 286, in decorated_function
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 360, in decorated_function
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 338, in decorated_function
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 326, in decorated_function
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 4321, in unshelve_instance
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     do_unshelve_instance()
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 445, in inner
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     return f(*args, **kwargs)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 4320, in do_unshelve_instance
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     filter_properties, node)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 4372, in _unshelve_instance
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     instance=instance)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/compute/manager.py", line 4368, in _unshelve_instance
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     block_device_info=block_device_info)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2342, in spawn
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     block_device_info)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/blockinfo.py", line 622, in get_disk_info
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     instance=instance)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/nova/nova/virt/libvirt/blockinfo.py", line 232, in get_disk_bus_for_device_type

  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher     disk_bus = image_meta.get('properties', {}).get(key)
  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher AttributeError: 'NoneType' object has no attribute 'get'

  2015-03-24 23:32:13.848 TRACE oslo_messaging.rpc.dispatcher
  2015-03-24 23:32:16.788 DEBUG nova.openstack.common.periodic_task [req-db2bb34f-1f3d-4ac0-99d0-f6fe78f8393d None None] Running periodic task ComputeManager._poll_rescued_instances from (pid=11271) run_periodic_tasks /opt/stack/nova/nova/openstack/common/periodic_task.py:219

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1436187/+subscriptions
References

[Bug 1436187] [NEW] 'AttributeError' is getting raised while unshelving instance booted from volume
From: Abhijeet Malawade, 2015-03-25