yahoo-eng-team team mailing list archive

[Lee Yarwood <1945608@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1945608

Title:
  Timed out waiting for server evacuate-bfv-test to go to ACTIVE status

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Description
  ===========

  Looks like a race between libvirtd restarting and the evacuation test
  running?

  https://zuul.opendev.org/t/openstack/build/c61ba346d3dc4677bb4a0a1520fec054/log/job-
  output.txt#62168

  2021-09-29 08:20:51.046250 | controller | + /opt/stack/nova/roles/run-
  evacuate-hook/files/test_evacuate.sh:evacuate_and_wait_for_active:34 :
  echo 'Timed out waiting for server evacuate-bfv-test to go to ACTIVE
  status'

  https://zuul.opendev.org/t/openstack/build/c61ba346d3dc4677bb4a0a1520fec054/log/controller/logs/screen-
  n-cpu.txt#11648

  Sep 29 08:09:48.352891 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: INFO nova.compute.manager [None req-a05f588d-c1e4-46d1-a129-539b9850302e demo admin] [instance: b0498f73-aaf2-436f-9db2-921774403901] Evacuating instance
  [..]
  Sep 29 08:09:48.444807 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: DEBUG nova.virt.libvirt.host [None req-a05f588d-c1e4-46d1-a129-539b9850302e demo admin] Connecting to libvirt: qemu:///system {{(pid=107626) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:499}}
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host [None req-a05f588d-c1e4-46d1-a129-539b9850302e demo admin] Connection to libvirt failed: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused: libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host Traceback (most recent call last):
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 586, in get_connection
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     conn = self._get_connection()
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 566, in _get_connection
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     self._queue_conn_event_handler(
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     self.force_reraise()
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     raise self.value
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 558, in _get_connection
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     self._wrapped_conn = self._get_new_connection()
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 502, in _get_new_connection
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     wrapped_conn = self._connect(self._uri, self._read_only)
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/opt/stack/nova/nova/virt/libvirt/host.py", line 353, in _connect
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     return self._libvirt_proxy.openAuth(uri, auth, flags)
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 193, in doit
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     result = proxy_call(self._autowrap, f, *args, **kwargs)
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 151, in proxy_call
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     rv = execute(f, *args, **kwargs)
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 132, in execute
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     six.reraise(c, e, tb)
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/six.py", line 719, in reraise
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     raise value
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/local/lib/python3.8/dist-packages/eventlet/tpool.py", line 86, in tworker
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     rv = meth(*args, **kwargs)
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host   File "/usr/lib/python3/dist-packages/libvirt.py", line 104, in openAuth
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host     if ret is None:raise libvirtError('virConnectOpenAuth() failed')
  Sep 29 08:09:48.456362 ubuntu-focal-iweb-mtl01-0026717282 nova-compute[107626]: ERROR nova.virt.libvirt.host libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1945608/+subscriptions