← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #05035

[Bug 1197922] Re: _sync_instance_power_state throws KeyError for shutdown instance

  Thread Previous • Date Previous • • Thread Next 
*** This bug is a duplicate of bug 1195849 ***
    https://bugs.launchpad.net/bugs/1195849

** This bug has been marked a duplicate of bug 1195849
   _sync_power_state fails when instance is powered off

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1197922

Title:
  _sync_instance_power_state throws KeyError for shutdown instance

Status in OpenStack Compute (Nova):
  Triaged

Bug description:
  Steps to reproduce in DevStack:

  1) Create an instance:  nova boot .....
  2) Shut it down with virsh:   sudo virsh shutdown n
  3) Wait for the compute manager to run the periodic power state sync (may take up to a minute).
  4) Compute manager triggers an Attribute Error on task_state as it tries to call stop() for the instance:

  
  2013-07-04 18:08:55.895 WARNING nova.compute.manager [-] [instance: 799f934d-2d05-455b-bf03-51b8c506e973] Instance shutdown by itself. Calling the stop API.
  2013-07-04 18:08:55.896 DEBUG nova.openstack.common.rpc.amqp [-] Making synchronous call on conductor ... from (pid=1367) multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:586
  2013-07-04 18:08:55.896 DEBUG nova.openstack.common.rpc.amqp [-] MSG_ID is ad7195d5326b4854b08aceb3fb7cc72c from (pid=1367) multicall /opt/stack/nova/nova/openstack/common/rpc/amqp.py:589
  2013-07-04 18:08:55.897 DEBUG nova.openstack.common.rpc.amqp [-] UNIQUE_ID is e2fef3b087734925afa50c0b68c03dad. from (pid=1367) _add_unique_id /opt/stack/nova/nova/openstack/common/rpc/amqp.py:337
  2013-07-04 18:08:56.056 DEBUG amqp [-] Closed channel #1 from (pid=1367) _do_close /usr/local/lib/python2.7/dist-packages/amqp/channel.py:88
  2013-07-04 18:08:56.057 DEBUG amqp [-] using channel_id: 1 from (pid=1367) __init__ /usr/local/lib/python2.7/dist-packages/amqp/channel.py:70
  2013-07-04 18:08:56.058 DEBUG amqp [-] Channel open from (pid=1367) _open_ok /usr/local/lib/python2.7/dist-packages/amqp/channel.py:420
  2013-07-04 18:08:56.058 ERROR nova.compute.manager [-] [instance: 799f934d-2d05-455b-bf03-51b8c506e973] error during stop() in sync_power_state.
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] Traceback (most recent call last):
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/compute/manager.py", line 4095, in _sync_instance_power_state
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     self.conductor_api.compute_stop(context, db_instance)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/conductor/api.py", line 332, in compute_stop
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return self._manager.compute_stop(context, instance, do_cast)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/conductor/rpcapi.py", line 460, in compute_stop
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return self.call(context, msg, version='1.43')
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/proxy.py", line 125, in call
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     result = rpc.call(context, real_topic, msg, timeout)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/__init__.py", line 140, in call
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return _get_impl().call(CONF, context, topic, msg, timeout)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/impl_kombu.py", line 798, in call
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     rpc_amqp.get_connection_pool(conf, Connection))
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 615, in call
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     rv = list(rv)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 564, in __iter__
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     raise result
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] AttributeError: 'dict' object has no attribute 'task_state'
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] Traceback (most recent call last):
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/amqp.py", line 433, in _process_data
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     **args)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     result = getattr(proxyobj, method)(ctxt, **kwargs)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/conductor/manager.py", line 508, in compute_stop
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     self.compute_api.stop(context, instance, do_cast)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/compute/api.py", line 179, in wrapped
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return func(self, context, target, *args, **kwargs)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/compute/api.py", line 169, in inner
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return function(self, context, instance, *args, **kwargs)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/compute/api.py", line 160, in wrapped
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return function(self, context, instance, *args, **kwargs)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/compute/api.py", line 150, in inner
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     return f(self, context, instance, *args, **kw)
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]   File "/opt/stack/nova/nova/compute/api.py", line 1396, in stop
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]     instance.task_state = task_states.POWERING_OFF
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] AttributeError: 'dict' object has no attribute 'task_state'
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973] 
  2013-07-04 18:08:56.058 TRACE nova.compute.manager [instance: 799f934d-2d05-455b-bf03-51b8c506e973]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1197922/+subscriptions
  Thread Previous • Date Previous • • Thread Next