yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: nova
       Status: Fix Committed => Fix Released

** Changed in: nova
    Milestone: None => liberty-3

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1381425

Title:
  nova cells, force-delete VM throws error even if VM gets deleted

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  In nova cells environment, when try to force-delete an instance which is in 'active' state, the instance gets deleted successfully, but in nova-cells service in compute cell (n-cell-child), it throws the following error:
  InvalidRequestError: Object '<Instance at 0x7fc5fcf581d0>' is already attached to session '75' (this is '79')

  Reproduction steps:
  1) Create instance.
  2) Wait until instance becomes 'active'.
  3) Try to force-delete the instance.
  $ nova force-delete <instance_id>

  Found this error in nova-cells service in compute cell (n-cell-child
  service):

  2014-10-15 01:59:36.742 ERROR nova.cells.messaging [req-7c1615ad-491d-4af8-88d7-ff83563ef429 admin admin] Error processing message locally: Object '<Ins
  tance at 0x7fc5fcf581d0>' is already attached to session '75' (this is '79')
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging Traceback (most recent call last):
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 199, in _process_locally
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     resp_value = self.msg_runner._process_message_locally(self)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 1293, in _process_message_locally
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     return fn(message, **message.method_kwargs)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 698, in run_compute_api_method
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     return fn(message.ctxt, *args, **method_info['method_kwargs'])
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 219, in wrapped
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     return func(self, context, target, *args, **kwargs)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 209, in inner
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     return function(self, context, instance, *args, **kwargs)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 190, in inner
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     return f(self, context, instance, *args, **kw)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 1836, in force_delete
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     self._delete_instance(context, instance)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 1790, in _delete_instance2014-10-15 01:59:36.742 TRACE nova.cells.messaging     task_state=task_states.DELETING)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 1622, in _delete
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     quotas.rollback()
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/local/lib/python2.7/dist-packages/oslo/utils/excutils.py", line 82, in __exit__
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     six.reraise(self.type_, self.value, self.tb)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/compute/api.py", line 1550, in _delete
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     instance.save()
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/db/sqlalchemy/models.py", line 52, in save
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     super(NovaBase, self).save(session=session)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/local/lib/python2.7/dist-packages/oslo/db/sqlalchemy/models.py", line 47, in save
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     session.add(self)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1399, in add
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     self._save_or_update_state(state)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1411, in _save_or_update_state
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     self._save_or_update_impl(state)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1667, in _save_or_update_impl
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     self._update_impl(state)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1661, in _update_impl
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     self._attach(state)
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging   File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1749, in _attach
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging     state.session_id, self.hash_key))
  2014-10-15 01:59:36.742 TRACE nova.cells.messaging InvalidRequestError: Object '<Instance at 0x7fc5fcf581d0>' is already attached to session '75' (this is '79')

  Note: This issue occurs intermitently.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1381425/+subscriptions