← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1489982] Re: Virt driver destroy exception is lost during instance shutdown if network deallocation fails

 

I created this bug based on old patches I had, a test program that
showed excutils.save_and_reraise_exception()  was not re-raising the
exception, and code inspection of the compute manager _shutdown_instance
method.

excutils.save_and_reraise_exception()  does not re-raise the exception,
but it does log it if you have a logger defined.  My test program did
not have a root logger defined so it did not log the exception.  With
the proper loggers defined, the original exception is logged with this
prefix "riginal exception being dropped:" and the exception that
occurred within the context manager is thrown, not the original
exception.

The secondary exception thrown from _try_deallocate_network is what will
be surfaced up the stack, but my concern about serviceability with a
lost exception stack is not valid.  Hence, I am closing out this defect.

Here is my test program and output that shows the original exception is
not rethrown but it is still logged.

[~]# cat test.py
from oslo_utils import excutils
import logging
import sys
logging.basicConfig()
sh = logging.StreamHandler(sys.stdout)
sh.setLevel(logging.DEBUG)
logging.getLogger('root').addHandler(sh)

try:
    raise Exception("Original")
except Exception:
    with excutils.save_and_reraise_exception():
        raise Exception("Second exception")

[~]# python test.py
ERROR:root:Original exception being dropped: ['Traceback (most recent call last):\n', '  File "test.py", line 10, in <module>\n    raise Exception("Original")\n', 'Exception: Original\n']
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    raise Exception("Second exception")
Exception: Second exception


** Changed in: nova
       Status: Incomplete => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1489982

Title:
  Virt driver destroy exception is lost during instance shutdown if
  network deallocation fails

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Version: OpenStack Liberty

  If the compute manager _shutdown_instance method's call to
  _try_deallocate_network at [1] fails, the exception from the virt
  driver destroy, which is the real root cause of the shutdown / delete
  instance failure is lost.

  This makes it harder to debug why the virt driver destroy method
  failed.

  [1]
  https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2252

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1489982/+subscriptions


References