← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #65911

[Bug 1704945] [NEW] nova force-delete can't delete instance in vm_state block_device_mapping

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Description
===========
when instance in block_device_mapping task state, can't force-delete it by nova force-delete command.

Steps to reproduce
==================
* Boot an instance, as soon as instance task state changed to block_device_mapping, run nova force-delete INSTANCE_UUID command, get some error:

# nova force-delete 08ed7e71-4470-4746-ae65-d17c659f29c9
/usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored.
  warnings.warn(msg)
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-b185ecd7-5f0d-441e-8263-59af8b66aae7)


Expected result
===============
force delete instance sucessfully

Actual result
=============
get some error and force delete instance failed

Environment
===========
# rpm -qa | grep nova
openstack-nova-api-15.0.0-1.el7.noarch
openstack-nova-console-15.0.0-1.el7.noarch
openstack-nova-common-15.0.0-1.el7.noarch
openstack-nova-conductor-15.0.0-1.el7.noarch
openstack-nova-novncproxy-15.0.0-1.el7.noarch
puppet-nova-10.3.0-1.el7.noarch
python-nova-15.0.0-1.el7.noarch
openstack-nova-cert-15.0.0-1.el7.noarch
openstack-nova-placement-api-15.0.0-1.el7.noarch
openstack-nova-compute-15.0.0-1.el7.noarch
openstack-nova-scheduler-15.0.0-1.el7.noarch
python2-novaclient-7.1.0-1.el7.noarch

Logs & Configs
==============
2017-07-18 15:47:22.159 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Action: 'action', calling method: <bound method DeferredDeleteController._force_delete of <nova.api.openstack.compute.deferred_delete.DeferredDeleteController object at 0x83f4c10>>, body: {"forceDelete": null} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:623
2017-07-18 15:47:22.159 36543 DEBUG nova.compute.api [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] [instance: 08ed7e71-4470-4746-ae65-d17c659f29c9] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2313
2017-07-18 15:47:22.172 36543 DEBUG oslo_db.sqlalchemy.engines [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:261
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Unexpected exception in API method
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/deferred_delete.py", line 64, in _force_delete
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     self.compute_api.force_delete(context, instance)
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 170, in inner
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     return function(self, context, instance, *args, **kwargs)
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 143, in inner
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     method=f.__name__)
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions InstanceInvalidState: Instance 08ed7e71-4470-4746-ae65-d17c659f29c9 in task_state block_device_mapping. Cannot force_delete while the instance is in this state.
2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions
2017-07-18 15:47:22.219 36543 INFO nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'>
2017-07-18 15:47:22.220 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'nova.exception.InstanceInvalidState'> __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1039
2017-07-18 15:47:22.221 36543 INFO nova.osapi_compute.wsgi.server [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] 192.168.2.145 "POST /v2.1/227c2369e7e9475f93c3ec7f07082937/servers/08ed7e71-4470-4746-ae65-d17c659f29c9/action HTTP/1.1" status: 500 len: 578 time: 0.0697980

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1704945

Title:
  nova force-delete can't delete instance in vm_state
  block_device_mapping

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  when instance in block_device_mapping task state, can't force-delete it by nova force-delete command.

  Steps to reproduce
  ==================
  * Boot an instance, as soon as instance task state changed to block_device_mapping, run nova force-delete INSTANCE_UUID command, get some error:

  # nova force-delete 08ed7e71-4470-4746-ae65-d17c659f29c9
  /usr/lib/python2.7/site-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored.
    warnings.warn(msg)
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception.InstanceInvalidState'> (HTTP 500) (Request-ID: req-b185ecd7-5f0d-441e-8263-59af8b66aae7)

  
  Expected result
  ===============
  force delete instance sucessfully

  Actual result
  =============
  get some error and force delete instance failed

  Environment
  ===========
  # rpm -qa | grep nova
  openstack-nova-api-15.0.0-1.el7.noarch
  openstack-nova-console-15.0.0-1.el7.noarch
  openstack-nova-common-15.0.0-1.el7.noarch
  openstack-nova-conductor-15.0.0-1.el7.noarch
  openstack-nova-novncproxy-15.0.0-1.el7.noarch
  puppet-nova-10.3.0-1.el7.noarch
  python-nova-15.0.0-1.el7.noarch
  openstack-nova-cert-15.0.0-1.el7.noarch
  openstack-nova-placement-api-15.0.0-1.el7.noarch
  openstack-nova-compute-15.0.0-1.el7.noarch
  openstack-nova-scheduler-15.0.0-1.el7.noarch
  python2-novaclient-7.1.0-1.el7.noarch

  Logs & Configs
  ==============
  2017-07-18 15:47:22.159 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Action: 'action', calling method: <bound method DeferredDeleteController._force_delete of <nova.api.openstack.compute.deferred_delete.DeferredDeleteController object at 0x83f4c10>>, body: {"forceDelete": null} _process_stack /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:623
  2017-07-18 15:47:22.159 36543 DEBUG nova.compute.api [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] [instance: 08ed7e71-4470-4746-ae65-d17c659f29c9] Fetching instance by UUID get /usr/lib/python2.7/site-packages/nova/compute/api.py:2313
  2017-07-18 15:47:22.172 36543 DEBUG oslo_db.sqlalchemy.engines [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:261
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Unexpected exception in API method
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions Traceback (most recent call last):
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 338, in wrapped
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/deferred_delete.py", line 64, in _force_delete
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     self.compute_api.force_delete(context, instance)
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 170, in inner
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     return function(self, context, instance, *args, **kwargs)
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 143, in inner
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions     method=f.__name__)
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions InstanceInvalidState: Instance 08ed7e71-4470-4746-ae65-d17c659f29c9 in task_state block_device_mapping. Cannot force_delete while the instance is in this state.
  2017-07-18 15:47:22.212 36543 ERROR nova.api.openstack.extensions
  2017-07-18 15:47:22.219 36543 INFO nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception.InstanceInvalidState'>
  2017-07-18 15:47:22.220 36543 DEBUG nova.api.openstack.wsgi [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'nova.exception.InstanceInvalidState'> __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1039
  2017-07-18 15:47:22.221 36543 INFO nova.osapi_compute.wsgi.server [req-b185ecd7-5f0d-441e-8263-59af8b66aae7 - - - - -] 192.168.2.145 "POST /v2.1/227c2369e7e9475f93c3ec7f07082937/servers/08ed7e71-4470-4746-ae65-d17c659f29c9/action HTTP/1.1" status: 500 len: 578 time: 0.0697980

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1704945/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next