← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #07887

[Bug 1265618] [NEW] image_snapshot_pending state breaks havana nodes

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Icehouse introduced a state called image_snapshot_pending which havana
nodes do not understand. If they call save with
expected_task_state="image_snapshot" they will crash on the new state.

2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2341, in _snapshot_instance
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     update_task_state)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1386, in snapshot
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     update_task_state(task_state=task_states.IMAGE_PENDING_UPLOAD)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2338, in update_task_state
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     instance.save(expected_task_state=expected_state)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/objects/base.py", line 139, in wrapper
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     ctxt, self, fn.__name__, args, kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 497, in object_action
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     objmethod=objmethod, args=args, kwargs=kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/rpcclient.py", line 85, in call
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return self._invoke(self.proxy.call, ctxt, method, **kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/rpcclient.py", line 63, in _invoke
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return cast_or_call(ctxt, msg, **self.kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/proxy.py", line 126, in call
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     result = rpc.call(context, real_topic, msg, timeout)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/__init__.py", line 139, in call
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return _get_impl().call(CONF, context, topic, msg, timeout)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     rpc_amqp.get_connection_pool(conf, Connection))
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 574, in call
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     rv = list(rv)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     raise result
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError_Remote: Unexpected task state: expecting (u'image_snapshot',) but the actual state is image_snapshot_pending
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/conductor/manager.py", line 576, in _object_dispatch
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return getattr(target, method)(context, *args, **kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/objects/base.py", line 152, in wrapper
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return fn(self, ctxt, *args, **kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/objects/instance.py", line 459, in save
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     columns_to_join=_expected_cols(expected_attrs))
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/api.py", line 735, in instance_update_and_get_original
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     columns_to_join=columns_to_join)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/sqlalchemy/api.py", line 130, in wrapper
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return f(*args, **kwargs)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/sqlalchemy/api.py", line 2200, in instance_update_and_get_original
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     columns_to_join=columns_to_join)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/sqlalchemy/api.py", line 2251, in _instance_update
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     actual=actual_state, expected=expected)
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError: Unexpected task state: expecting (u'image_snapshot',) but the actual state is image_snapshot_pending
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp

** Affects: nova
     Importance: Undecided
         Status: Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1265618

Title:
  image_snapshot_pending state breaks havana nodes

Status in OpenStack Compute (Nova):
  Confirmed

Bug description:
  Icehouse introduced a state called image_snapshot_pending which havana
  nodes do not understand. If they call save with
  expected_task_state="image_snapshot" they will crash on the new state.

  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2341, in _snapshot_instance
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     update_task_state)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1386, in snapshot
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     update_task_state(task_state=task_states.IMAGE_PENDING_UPLOAD)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/compute/manager.py", line 2338, in update_task_state
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     instance.save(expected_task_state=expected_state)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/objects/base.py", line 139, in wrapper
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     ctxt, self, fn.__name__, args, kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 497, in object_action
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     objmethod=objmethod, args=args, kwargs=kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/rpcclient.py", line 85, in call
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return self._invoke(self.proxy.call, ctxt, method, **kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/rpcclient.py", line 63, in _invoke
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return cast_or_call(ctxt, msg, **self.kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/proxy.py", line 126, in call
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     result = rpc.call(context, real_topic, msg, timeout)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/__init__.py", line 139, in call
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return _get_impl().call(CONF, context, topic, msg, timeout)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/impl_kombu.py", line 816, in call
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     rpc_amqp.get_connection_pool(conf, Connection))
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 574, in call
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     rv = list(rv)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova-havana/local/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 539, in __iter__
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     raise result
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError_Remote: Unexpected task state: expecting (u'image_snapshot',) but the actual state is image_snapshot_pending
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/conductor/manager.py", line 576, in _object_dispatch
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return getattr(target, method)(context, *args, **kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/objects/base.py", line 152, in wrapper
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return fn(self, ctxt, *args, **kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/objects/instance.py", line 459, in save
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     columns_to_join=_expected_cols(expected_attrs))
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/api.py", line 735, in instance_update_and_get_original
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     columns_to_join=columns_to_join)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/sqlalchemy/api.py", line 130, in wrapper
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     return f(*args, **kwargs)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/sqlalchemy/api.py", line 2200, in instance_update_and_get_original
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     columns_to_join=columns_to_join)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp   File "/opt/upstack/nova/nova/db/sqlalchemy/api.py", line 2251, in _instance_update
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp     actual=actual_state, expected=expected)
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp UnexpectedTaskStateError: Unexpected task state: expecting (u'image_snapshot',) but the actual state is image_snapshot_pending
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp 
  2014-01-02 11:58:46.766 TRACE nova.openstack.common.rpc.amqp

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1265618/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next