yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1687086] [NEW] nova fails to rescue an instance because ramdisk file doesn't exist

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Ihar Hrachyshka <1687086@xxxxxxxxxxxxxxxxxx>
Date: Fri, 28 Apr 2017 18:14:48 -0000
Reply-to: Bug 1687086 <1687086@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

http://logs.openstack.org/67/457467/4/gate/gate-tempest-dsvm-neutron-
dvr-ubuntu-
xenial/4d6be0a/logs/screen-n-cpu.txt.gz?level=TRACE#_2017-04-20_16_18_59_065

2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [req-26543eff-dd70-4526-bec6-fc977ea734dc tempest-ServerRescueNegativeTestJSON-295821689 tempest-ServerRescueNegativeTestJSON-295821689] [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] Error trying to Rescue Instance
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] Traceback (most recent call last):
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 3370, in rescue_instance
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     rescue_image_meta, admin_password)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2636, in rescue
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     self._create_domain(xml, post_xml_callback=gen_confdrive)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 5002, in _create_domain
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     guest.launch(pause=pause)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 145, in launch
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     self._encoded_xml, errors='ignore')
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     self.force_reraise()
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     six.reraise(self.type_, self.value, self.tb)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 140, in launch
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     return self._domain.createWithFlags(flags)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     rv = execute(f, *args, **kwargs)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     six.reraise(c, e, tb)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     rv = meth(*args, **kwargs)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 1065, in createWithFlags
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] libvirtError: unable to stat: /opt/stack/data/nova/instances/6e63ccaa-f174-4371-a169-d5303db821eb/ramdisk.rescue: No such file or directory
2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] 
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server [req-26543eff-dd70-4526-bec6-fc977ea734dc tempest-ServerRescueNegativeTestJSON-295821689 tempest-ServerRescueNegativeTestJSON-295821689] Exception during message handling
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 75, in wrapped
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 66, in wrapped
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 188, in decorated_function
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     LOG.warning(msg, e, instance=instance)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 157, in decorated_function
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/utils.py", line 686, in decorated_function
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 216, in decorated_function
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     self.force_reraise()
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 204, in decorated_function
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 3377, in rescue_instance
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     reason=_("Driver Error: %s") % e)
2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server InstanceNotRescuable: Instance 6e63ccaa-f174-4371-a169-d5303db821eb cannot be rescued: Driver Error: unable to stat: /opt/stack/data/nova/instances/6e63ccaa-f174-4371-a169-d5303db821eb/ramdisk.rescue: No such file or directory

This seems to happen for multiple instances in the same test run.
Nothing suspicious that I could spot in libvirt logs.

It's Ocata but I am not sure it's not present in master.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: gate-failure

** Tags added: gate-failure

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1687086

Title:
  nova fails to rescue an instance because ramdisk file doesn't exist

Status in OpenStack Compute (nova):
  New

Bug description:
  http://logs.openstack.org/67/457467/4/gate/gate-tempest-dsvm-neutron-
  dvr-ubuntu-
  xenial/4d6be0a/logs/screen-n-cpu.txt.gz?level=TRACE#_2017-04-20_16_18_59_065

  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [req-26543eff-dd70-4526-bec6-fc977ea734dc tempest-ServerRescueNegativeTestJSON-295821689 tempest-ServerRescueNegativeTestJSON-295821689] [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] Error trying to Rescue Instance
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] Traceback (most recent call last):
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/compute/manager.py", line 3370, in rescue_instance
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     rescue_image_meta, admin_password)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2636, in rescue
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     self._create_domain(xml, post_xml_callback=gen_confdrive)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 5002, in _create_domain
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     guest.launch(pause=pause)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 145, in launch
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     self._encoded_xml, errors='ignore')
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     self.force_reraise()
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     six.reraise(self.type_, self.value, self.tb)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 140, in launch
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     return self._domain.createWithFlags(flags)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     rv = execute(f, *args, **kwargs)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     six.reraise(c, e, tb)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     rv = meth(*args, **kwargs)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]   File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 1065, in createWithFlags
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] libvirtError: unable to stat: /opt/stack/data/nova/instances/6e63ccaa-f174-4371-a169-d5303db821eb/ramdisk.rescue: No such file or directory
  2017-04-20 16:18:59.065 2166 ERROR nova.compute.manager [instance: 6e63ccaa-f174-4371-a169-d5303db821eb] 
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server [req-26543eff-dd70-4526-bec6-fc977ea734dc tempest-ServerRescueNegativeTestJSON-295821689 tempest-ServerRescueNegativeTestJSON-295821689] Exception during message handling
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 155, in _process_incoming
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 222, in dispatch
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 192, in _do_dispatch
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 75, in wrapped
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/exception_wrapper.py", line 66, in wrapped
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 188, in decorated_function
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     LOG.warning(msg, e, instance=instance)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 157, in decorated_function
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/utils.py", line 686, in decorated_function
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 216, in decorated_function
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     self.force_reraise()
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 204, in decorated_function
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server   File "/opt/stack/new/nova/nova/compute/manager.py", line 3377, in rescue_instance
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server     reason=_("Driver Error: %s") % e)
  2017-04-20 16:18:59.523 2166 ERROR oslo_messaging.rpc.server InstanceNotRescuable: Instance 6e63ccaa-f174-4371-a169-d5303db821eb cannot be rescued: Driver Error: unable to stat: /opt/stack/data/nova/instances/6e63ccaa-f174-4371-a169-d5303db821eb/ramdisk.rescue: No such file or directory

  This seems to happen for multiple instances in the same test run.
  Nothing suspicious that I could spot in libvirt logs.

  It's Ocata but I am not sure it's not present in master.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1687086/+subscriptions
Follow ups

[Bug 1687086] Re: nova fails to rescue an instance because ramdisk file doesn't exist
From: Launchpad Bug Tracker, 2017-09-26