openstack team mailing list archive

Thread
Date
Terminating instance stuck at "deleting" - Openstack on Fedora Core 17

To: openstack@xxxxxxxxxxxxxxxxxxx
From: Nandakumar Pezhery <nandakumarpr@xxxxxxxxx>
Date: Wed, 26 Sep 2012 23:43:18 +0400
Hi Everyone,
I've been working on the FC 17 version of Openstack for few days - single
box implementation.

Request help to fix the below problem which seems to have started after I
configured floating IPs.

In nova.conf,
auto_assign_floating_ip=True

I get the below error in compute.log while terminating an instance (Status
= Active, Task = deleting, Power state = running).
====================================================
2012-09-26 23:11:11 INFO nova.compute.manager [-] Updating host status
2012-09-26 23:11:58 INFO nova.compute.manager
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] check_instance_lock: decorating:
|<function terminate_instance at 0x1725a28>|
2012-09-26 23:11:58 INFO nova.compute.manager
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] check_instance_lock: arguments:
|<nova.compute.manager.ComputeManager object at 0xc93bd0>|
|<nova.rpc.amqp.RpcContext object at 0x323bc90>|
|072fe1b9-904e-4dd9-8624-2860c2e7bef6|
2012-09-26 23:11:58 INFO nova.compute.manager
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] check_instance_lock: locked: |False|
2012-09-26 23:11:58 INFO nova.compute.manager
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] check_instance_lock: admin: |True|
2012-09-26 23:11:58 INFO nova.compute.manager
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] check_instance_lock: executing:
|<function terminate_instance at 0x1725a28>|
2012-09-26 23:11:58 AUDIT nova.compute.manager
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] [instance:
072fe1b9-904e-4dd9-8624-2860c2e7bef6] Terminating instance
2012-09-26 23:11:58 ERROR nova.rpc.amqp
[req-0076990a-3037-483b-a60f-9dee2cff362a 53f453879d1245159592afb8380d8607
833336f7c6c14bc4bc2134ccedae5c77] Exception during message handling
2012-09-26 23:11:58 TRACE nova.rpc.amqp Traceback (most recent call last):
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 253, in
_process_data
2012-09-26 23:11:58 TRACE nova.rpc.amqp     rval = node_func(context=ctxt,
**node_args)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/exception.py", line 114, in wrapped
2012-09-26 23:11:58 TRACE nova.rpc.amqp     return f(*args, **kw)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 159, in
decorated_function
2012-09-26 23:11:58 TRACE nova.rpc.amqp     function(self, context,
instance_uuid, *args, **kwargs)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 183, in
decorated_function
2012-09-26 23:11:58 TRACE nova.rpc.amqp     sys.exc_info())
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
2012-09-26 23:11:58 TRACE nova.rpc.amqp     self.gen.next()
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 177, in
decorated_function
2012-09-26 23:11:58 TRACE nova.rpc.amqp     return function(self, context,
instance_uuid, *args, **kwargs)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 771, in
terminate_instance
2012-09-26 23:11:58 TRACE nova.rpc.amqp     do_terminate_instance()
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/utils.py", line 946, in inner
2012-09-26 23:11:58 TRACE nova.rpc.amqp     retval = f(*args, **kwargs)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 764, in
do_terminate_instance
2012-09-26 23:11:58 TRACE nova.rpc.amqp     self._delete_instance(context,
instance)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 742, in
_delete_instance
2012-09-26 23:11:58 TRACE nova.rpc.amqp
self._shutdown_instance(context, instance, 'Terminating')
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 704, in
_shutdown_instance
2012-09-26 23:11:58 TRACE nova.rpc.amqp
self._deallocate_network(context, instance)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 636, in
_deallocate_network
2012-09-26 23:11:58 TRACE nova.rpc.amqp
self.network_api.deallocate_for_instance(context, instance)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/network/api.py", line 190, in
deallocate_for_instance
2012-09-26 23:11:58 TRACE nova.rpc.amqp     'args': args})
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/rpc/__init__.py", line 68, in call
2012-09-26 23:11:58 TRACE nova.rpc.amqp     return
_get_impl().call(context, topic, msg, timeout)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 526, in call
2012-09-26 23:11:58 TRACE nova.rpc.amqp     return rpc_amqp.call(context,
topic, msg, timeout, Connection.pool)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 343, in call
2012-09-26 23:11:58 TRACE nova.rpc.amqp     rv = list(rv)
2012-09-26 23:11:58 TRACE nova.rpc.amqp   File
"/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 311, in __iter__
2012-09-26 23:11:58 TRACE nova.rpc.amqp     raise result
2012-09-26 23:11:58 TRACE nova.rpc.amqp RemoteError: Remote error:
NotAuthorized Not authorized.
2012-09-26 23:11:58 TRACE nova.rpc.amqp [u'Traceback (most recent call
last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py",
line 253, in _process_data\n    rval = node_func(context=ctxt,
**node_args)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/network/manager.py", line 257, in
wrapped\n    return func(self, context, *args, **kwargs)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/network/manager.py", line 370, in
deallocate_for_instance\n    affect_auto_assigned=True)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/network/manager.py", line 257, in
wrapped\n    return func(self, context, *args, **kwargs)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/network/manager.py", line 514, in
disassociate_floating_ip\n    self._floating_ip_owned_by_project(context,
floating_ip)\n', u'  File
"/usr/lib/python2.7/site-packages/nova/network/manager.py", line 387, in
_floating_ip_owned_by_project\n    raise exception.NotAuthorized()\n',
u'NotAuthorized: Not authorized.\n'].
2012-09-26 23:11:58 TRACE nova.rpc.amqp
=====================================
Follow ups

Re: Terminating instance stuck at "deleting" - Openstack on Fedora Core 17
From: Vishvananda Ishaya, 2012-09-26