yahoo-eng-team team mailing list archive

[Rajesh Tailor <ratailor@xxxxxxxxxx>]

Public bug reported:

Problem Description:

When user tries to force-delete instance which is not in task_state
'None', throws 500 error on console, as below:

ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8)

Steps to reproduce:
1) Create instance.
$ nova boot --flavor <flavor-id> --image <image-id> <instance-name>

2) To change the instance task_state to anything other than 'None', I tried to resize the instance.
$ nova resize <instance-id> <flavor-id>

3) Try to force-delete instance, just after above step, so that instance task state is anything in (resize_prep, resize_migrating, resize_migrated, resize_finish), but not 'None'.
$ nova force-delete <instance-id>

4) User gets below error on console:

ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8)


Actual result: 
User gets 500 ClientException Error.

Expected result:
The instance should be deleted without any error.

traceback from nova-api logs:

DEBUG oslo_concurrency.lockutils [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Lock "fbe6eec8-be64-4473-8434-1d795e7ca5fb" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.0
00s {{(pid=24369) inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285}}
ERROR nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Unexpected exception in API method: InstanceInvalidState: Instance 6abb021a-174e-4551-acc1-a96653a9bf83 in task_state resize_prep. Cannot force_delete while the instance is in this state.
ERROR nova.api.openstack.wsgi Traceback (most recent call last):
ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 803, in wrapped
ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 61, in _force_delete
ERROR nova.api.openstack.wsgi     self.compute_api.force_delete(context, instance)
ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/compute/api.py", line 201, in inner
ERROR nova.api.openstack.wsgi     return function(self, context, instance, *args, **kwargs)
ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/compute/api.py", line 141, in inner
ERROR nova.api.openstack.wsgi     method=f.__name__)
ERROR nova.api.openstack.wsgi InstanceInvalidState: Instance 6abb021a-174e-4551-acc1-a96653a9bf83 in task_state resize_prep. Cannot force_delete while the instance is in this state.
ERROR nova.api.openstack.wsgi 
INFO nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'>
DEBUG nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.


NOTE: If user tries delete api instead of force-delete api for instance deletion, in that case instance is deleted and no error is thrown.

** Affects: nova
     Importance: Undecided
     Assignee: Rajesh Tailor (ratailor)
         Status: In Progress

** Changed in: nova
     Assignee: (unassigned) => Rajesh Tailor (ratailor)

** Changed in: nova
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1741000

Title:
  Can't force-delete instance with task_state not 'None'

Status in OpenStack Compute (nova):
  In Progress

Bug description:
  Problem Description:

  When user tries to force-delete instance which is not in task_state
  'None', throws 500 error on console, as below:

  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8)

  Steps to reproduce:
  1) Create instance.
  $ nova boot --flavor <flavor-id> --image <image-id> <instance-name>

  2) To change the instance task_state to anything other than 'None', I tried to resize the instance.
  $ nova resize <instance-id> <flavor-id>

  3) Try to force-delete instance, just after above step, so that instance task state is anything in (resize_prep, resize_migrating, resize_migrated, resize_finish), but not 'None'.
  $ nova force-delete <instance-id>

  4) User gets below error on console:

  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8)

  
  Actual result: 
  User gets 500 ClientException Error.

  Expected result:
  The instance should be deleted without any error.

  traceback from nova-api logs:

  DEBUG oslo_concurrency.lockutils [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Lock "fbe6eec8-be64-4473-8434-1d795e7ca5fb" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.0
  00s {{(pid=24369) inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285}}
  ERROR nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Unexpected exception in API method: InstanceInvalidState: Instance 6abb021a-174e-4551-acc1-a96653a9bf83 in task_state resize_prep. Cannot force_delete while the instance is in this state.
  ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 803, in wrapped
  ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 61, in _force_delete
  ERROR nova.api.openstack.wsgi     self.compute_api.force_delete(context, instance)
  ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/compute/api.py", line 201, in inner
  ERROR nova.api.openstack.wsgi     return function(self, context, instance, *args, **kwargs)
  ERROR nova.api.openstack.wsgi   File "/opt/stack/nova/nova/compute/api.py", line 141, in inner
  ERROR nova.api.openstack.wsgi     method=f.__name__)
  ERROR nova.api.openstack.wsgi InstanceInvalidState: Instance 6abb021a-174e-4551-acc1-a96653a9bf83 in task_state resize_prep. Cannot force_delete while the instance is in this state.
  ERROR nova.api.openstack.wsgi 
  INFO nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception.InstanceInvalidState'>
  DEBUG nova.api.openstack.wsgi [None req-2578def4-a83d-458c-a3cb-c5fa1e6b56a8 demo demo] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.

  
  NOTE: If user tries delete api instead of force-delete api for instance deletion, in that case instance is deleted and no error is thrown.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1741000/+subscriptions