yahoo-eng-team team mailing list archive

[Balazs Gibizer <1915364@xxxxxxxxxxxxxxxxxx>]

I guess this means that the libvirtd service isn't running on your compute hosts. The 'org.freedesktop.machine1' points to systemd or dbus issues on your hypervisor[1][2][3]. I think as a next step you should check the status of your systemd-machined service.
 
sudo systemctl status systemd-machined.service

I'm closing this report as it is not a Nova bug.

[1] https://man7.org/linux/man-pages/man5/org.freedesktop.machine1.5.html
[2] https://www.freedesktop.org/wiki/Software/systemd/machined/
[3] https://listman.redhat.com/archives/libvirt-users/2017-July/msg00034.html

** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1915364

Title:
  fail to launch instances

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  I'm testing openstack on my home lab (3x Dell T410 loaded).  I've been
  using lately openstack-ansible to deploy.  Besides some issues here
  and there, the overall process succeed.  I'm presently hitting an
  issue that I can't seem to find the possible cause.

  Exhausted all hosts available for retrying ... I think this is a
  general response always giving even if the issue is with the network,
  storage, resource, etc.

  FYI, I'm using ceph as a backend and it works good.
  I'm able to create volumes, images, volumes-from-images, all from openstack command line and horizon.
  I have networks which I can ping from outside openstack. 
  All openstack services shows ok.  From the logs, I seem to have some issues with gnocchi, I don't know to what extend this coud prevents me from spawning instances.

  gnocchi.incoming.SackDetectionError: [Errno 2] No such file or
  directory: '/var/lib/gnocchi/tmp/gnocchi-config'

  I have full logs, but I can't seem to find the culprit.

  In the compute-node, I have this..pretty cryptic...:

  ERROR nova.compute.manager [req-66f25425-c735-44dd-aa69-d8e1c904f17e 0540dc5ba00641c1b86a269b99d4b323 ddc8fdea71a943a7b36c4ae66b2f44ed - default default] [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95] Failed to build and run instance: libvirt.libvirtError: Failed to activate service 'org.freedesktop.machine1': timed out (service_start_timeout=25000ms)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95] Traceback (most recent call last):#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/compute/manager.py", line 2384, in _build_and_run_instance#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     self.driver.spawn(context, instance, image_meta,#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 3682, in spawn#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     self._create_guest_with_network(#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 6683, in _create_guest_with_network#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     self._cleanup_failed_start(#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     self.force_reraise()#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     six.reraise(self.type_, self.value, self.tb)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/six.py", line 703, in reraise#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     raise value#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 6652, in _create_guest_with_network#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     guest = self._create_guest(#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/virt/libvirt/driver.py", line 6585, in _create_guest#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     guest.launch(pause=pause)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/virt/libvirt/guest.py", line 158, in launch#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     LOG.exception('Error launching a defined domain with XML: %s',#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/oslo_utils/excutils.py", line 220, in __exit__#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     self.force_reraise()#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/oslo_utils/excutils.py", line 196, in force_reraise#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     six.reraise(self.type_, self.value, self.tb)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/six.py", line 703, in reraise#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     raise value#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/nova/virt/libvirt/guest.py", line 155, in launch#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     return self._domain.createWithFlags(flags)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/eventlet/tpool.py", line 190, in doit#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     result = proxy_call(self._autowrap, f, *args, **kwargs)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/eventlet/tpool.py", line 148, in proxy_call#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     rv = execute(f, *args, **kwargs)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/eventlet/tpool.py", line 129, in execute#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     six.reraise(c, e, tb)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/six.py", line 703, in reraise#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     raise value#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/eventlet/tpool.py", line 83, in tworker#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     rv = meth(*args, **kwargs)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]   File "/openstack/venvs/nova-22.0.0.0rc2.dev1/lib/python3.8/site-packages/libvirt.py", line 1265, in createWithFlags#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)#0122021-02-09 09:41:39.705 408599
  ERROR nova.compute.manager [instance: b721cf9a-e3f3-4b89-8750-1b9f1e273f95] libvirt.libvirtError: Failed to activate service 'org.freedesktop.machine1': timed out (service_start_timeout=25000ms)#0122021-02-09 09:41:39.705 408599

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1915364/+subscriptions