yahoo-eng-team team mailing list archive

Thread
Date

[Bug 1788922] [NEW] SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date: Fri, 24 Aug 2018 19:05:08 -0000
Reply-to: Bug 1788922 <1788922@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx

Public bug reported:

Seen here:

http://logs.openstack.org/71/594571/2/gate/nova-tox-functional-
py35/fd2d9ac/testr_results.html.gz

2018-08-24 16:36:47,192 ERROR [nova.compute.manager] Instance failed to spawn
Traceback (most recent call last):
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/compute/manager.py", line 2354, in _build_resources
    yield resources
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/compute/manager.py", line 2118, in _build_and_run_instance
    block_device_info=block_device_info)
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 3075, in spawn
    mdevs=mdevs)
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 5430, in _get_guest_xml
    context, mdevs)
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 5216, in _get_guest_config
    flavor, guest.os_type)
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 3995, in _get_guest_storage_config
    inst_type)
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 3903, in _get_guest_disk_config
    self.disk_cachemode,
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 416, in disk_cachemode
    if not nova.privsep.utils.supports_direct_io(CONF.instances_path):
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/privsep/utils.py", line 62, in supports_direct_io
    {'path': dirpath, 'ex': e})
  File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py35/lib/python3.5/site-packages/oslo_utils/excutils.py", line 220, in __exit__
    self.force_reraise()
  File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py35/lib/python3.5/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py35/lib/python3.5/site-packages/six.py", line 693, in reraise
    raise value
  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/privsep/utils.py", line 45, in supports_direct_io
    fd = os.open(testfile, os.O_CREAT | os.O_WRONLY | os.O_DIRECT)
FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22FileNotFoundError%3A%20%5BErrno%202%5D%20No%20such%20file%20or%20directory%3A%20'%2Fhome%2Fzuul%2Fsrc%2Fgit.openstack.org%2Fopenstack%2Fnova%2Finstances%2F.directio.test'%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d

Just started, so it's likely related to these changes:

https://review.openstack.org/#/c/595802/
https://review.openstack.org/#/c/407055/

** Affects: nova
     Importance: Medium
         Status: Confirmed


** Tags: functional libvirt

** Changed in: nova
       Status: New => Confirmed

** Changed in: nova
   Importance: Undecided => Medium

** Tags added: functional libvirt

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1788922

Title:
  SRIOVServersTest.test_create_server_with_VF intermittently fails due
  to "FileNotFoundError: [Errno 2] No such file or directory:
  '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  Seen here:

  http://logs.openstack.org/71/594571/2/gate/nova-tox-functional-
  py35/fd2d9ac/testr_results.html.gz

  2018-08-24 16:36:47,192 ERROR [nova.compute.manager] Instance failed to spawn
  Traceback (most recent call last):
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/compute/manager.py", line 2354, in _build_resources
      yield resources
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/compute/manager.py", line 2118, in _build_and_run_instance
      block_device_info=block_device_info)
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 3075, in spawn
      mdevs=mdevs)
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 5430, in _get_guest_xml
      context, mdevs)
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 5216, in _get_guest_config
      flavor, guest.os_type)
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 3995, in _get_guest_storage_config
      inst_type)
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 3903, in _get_guest_disk_config
      self.disk_cachemode,
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/virt/libvirt/driver.py", line 416, in disk_cachemode
      if not nova.privsep.utils.supports_direct_io(CONF.instances_path):
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/privsep/utils.py", line 62, in supports_direct_io
      {'path': dirpath, 'ex': e})
    File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py35/lib/python3.5/site-packages/oslo_utils/excutils.py", line 220, in __exit__
      self.force_reraise()
    File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py35/lib/python3.5/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py35/lib/python3.5/site-packages/six.py", line 693, in reraise
      raise value
    File "/home/zuul/src/git.openstack.org/openstack/nova/nova/privsep/utils.py", line 45, in supports_direct_io
      fd = os.open(testfile, os.O_CREAT | os.O_WRONLY | os.O_DIRECT)
  FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'

  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22FileNotFoundError%3A%20%5BErrno%202%5D%20No%20such%20file%20or%20directory%3A%20'%2Fhome%2Fzuul%2Fsrc%2Fgit.openstack.org%2Fopenstack%2Fnova%2Finstances%2F.directio.test'%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d

  Just started, so it's likely related to these changes:

  https://review.openstack.org/#/c/595802/
  https://review.openstack.org/#/c/407055/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1788922/+subscriptions

Follow ups

[Bug 1788922] Re: SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"
From: OpenStack Infra, 2018-08-27