yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1521823] Re: reboot test fails in gate-grenade-dsvm-multinode with missing disk path

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem@xxxxxxxxxx>
Date: Fri, 13 May 2016 15:15:40 -0000
Reply-to: Bug 1521823 <1521823@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Fixed by infra.

** Changed in: nova
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1521823

Title:
  reboot test fails in gate-grenade-dsvm-multinode with missing disk
  path

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  http://logs.openstack.org/44/245344/4/gate/gate-grenade-dsvm-
  multinode/1840523/logs/subnode-2/old/screen-n-cpu.txt.gz?level=TRACE#_2015-12-01_23_58_04_999

  2015-12-01 23:58:04.999 ERROR nova.compute.manager [req-8e76a5ba-f89f-4bfe-8218-aeaa580a6e13 tempest-ServerActionsTestJSON-475123373 tempest-ServerActionsTestJSON-884217194] [instance: e50d2ceb-2be4-4c1a-a190-4aa9ab160af9] Cannot reboot instance: [Errno 2] No such file or directory: '/opt/stack/data/nova/instances/e50d2ceb-2be4-4c1a-a190-4aa9ab160af9/disk'
  2015-12-01 23:58:05.513 ERROR oslo_messaging.rpc.dispatcher [req-8e76a5ba-f89f-4bfe-8218-aeaa580a6e13 tempest-ServerActionsTestJSON-475123373 tempest-ServerActionsTestJSON-884217194] Exception during message handling: [Errno 2] No such file or directory: '/opt/stack/data/nova/instances/e50d2ceb-2be4-4c1a-a190-4aa9ab160af9/disk'
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     executor_callback))
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     executor_callback)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/exception.py", line 89, in wrapped
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     payload)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/exception.py", line 72, in wrapped
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 350, in decorated_function
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     LOG.warning(msg, e, instance=instance)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 323, in decorated_function
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 400, in decorated_function
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 378, in decorated_function
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 366, in decorated_function
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 2954, in reboot_instance
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     self._set_instance_obj_error_state(context, instance)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/compute/manager.py", line 2935, in reboot_instance
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     bad_volumes_callback=bad_volumes_callback)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 2055, in reboot
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     block_device_info)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 2152, in _hard_reboot
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     block_device_info)
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/opt/stack/old/nova/nova/virt/libvirt/driver.py", line 6484, in _get_instance_disk_info
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     dk_size = int(os.path.getsize(path))
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/genericpath.py", line 49, in getsize
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher     return os.stat(filename).st_size
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher OSError: [Errno 2] No such file or directory: '/opt/stack/data/nova/instances/e50d2ceb-2be4-4c1a-a190-4aa9ab160af9/disk'
  2015-12-01 23:58:05.513 31663 ERROR oslo_messaging.rpc.dispatcher 

  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22Cannot%20reboot%20instance:%20%5BErrno%202%5D%20No%20such%20file%20or%20directory%5C%22%20AND%20tags:%5C%22screen-n-cpu.txt%5C%22

  122 hits in 7 days, check and gate, master and stable/liberty, all
  failures.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1521823/+subscriptions
References

[Bug 1521823] [NEW] reboot test fails in gate-grenade-dsvm-multinode with missing disk path
From: Matt Riedemann, 2015-12-02