yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1963553] [NEW] Openstack Fails to Launch Instances "/usr/bin/qemu-system-arm' does not support virt type 'kvm; "

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Evan <1963553@xxxxxxxxxxxxxxxxxx>
Date: Thu, 03 Mar 2022 21:08:54 -0000
Reply-to: Bug 1963553 <1963553@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Environment - Openstack Victoria on Ubuntu 20.0.4 on Raspberry Pi 4B, 1
controller, 2 compute, 1 storage nodes.

Been troubleshooting an Raspberry Pi 4B Openstack setup. I have all my
openstack components running and verified but when trying to launch an
instance it errors out (seems to be from libvirt):

 Emulator '/usr/bin/qemu-system-arm' does not support virt type
'kvm'\n", '\nDuring handling of the above exception, another exception
occurred

Per these instructions
(https://docs.openstack.org/nova/victoria/configuration/config.html) as
a fix I tried I've tried the following options in nova.conf in all
variations with no difference in outcome (Restarted Libvirt and Nova-
Computer service after each change as well)

cpu_mode = (default)

cpu_mode = host-passthrough

virt_type = kvm

virt_type = qemu

I'm at a loss for what to do as to my knowledge the only way information
is passed through to libvirt is through nova.conf, and appreciate any
assistance.

Controller Node /var/log/nova-conductor.log

    2022-03-03 16:39:13.462 1987 ERROR nova.scheduler.utils [req-de8dd1d0-3ff5-4495-ad8b-8af235b4d8c4 d20aa0616f264b39a2b72422d2d5d947 - - default default] [instance: c453aedc-08d5-4b5c-95c9-ddda1eab4514] Error from last host: compute2 (node compute2): ['Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2442, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3766, in spawn\n self._create_guest_with_network(\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6758, in _create_guest_with_network\n
    self.cleanup_failed_start(\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in exit\n self.force_reraise()\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type, self.value, self.tb)\n', ' File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise\n
    raise value\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6727, in _create_guest_with_network\n guest = self._create_guest(\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6655, in _create_guest\n guest = libvirt_guest.Guest.create(xml, self.host)\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 144, in create\n LOG.error('Error defining a guest with XML: %s',\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in exit\n self.force_reraise()\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type, self.value, self.tb)\n', ' File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise\n
    raise value\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 141, in create\n guest = host.write_instance_config(xml)\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/host.py", line 1144, in write_instance_config\n domain = self.get_connection().defineXML(xml)\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 193, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 151, in proxy_call\n rv = execute(f, *args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 132, in execute\n six.reraise(c, e, tb)\n', ' File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise\n
    raise value\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 86, in tworker\n rv = meth(*args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/libvirt.py", line 4047, in defineXML\n if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)\n', "libvirt.libvirtError: unsupported configuration: Emulator '/usr/bin/qemu-system-arm' does not support virt type 'kvm'\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2268, in _do_build_and_run_instance\n
    self._build_and_run_instance(context, instance, image,\n', ' File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2538, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance c453aedc-08d5-4b5c-95c9-ddda1eab4514 was re-scheduled: unsupported configuration: Emulator '/usr/bin/qemu-system-arm' does not support virt type 'kvm'\n"] 2022-03-03 16:39:13.466 1987 WARNING nova.scheduler.utils [req-de8dd1d0-3ff5-4495-ad8b-8af235b4d8c4 d20aa0616f264b39a2b72422d2d5d947 - - default default] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c453aedc-08d5-4b5c-95c9-ddda1eab4514.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c453aedc-08d5-4b5c-95c9-ddda1eab4514.

Compute Node libvirtd debug log

    2022-01-10 04:58:27.670+0000: 809: error : virCPUGetHost:439 : this
function is not supported by the connection driver: cannot detect host
CPU model for armv7l architecture 2022-01-10 04:58:27.670+0000: 809:
error : virCPUGetHost:439 : this function is not supported by the
connection driver: cannot detect host CPU model for armv7l architecture
2022-01-10 04:58:30.362+0000: 809: error :
virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
04:58:30.370+0000: 807: error : virQEMUCapsCacheLookupDefault:5577 :
invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on
this host 2022-01-10 04:58:30.379+0000: 808: error :
virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
04:58:30.387+0000: 806: error : virQEMUCapsCacheLookupDefault:5577 :
invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64'
on this host 2022-01-10 04:58:30.393+0000: 810: error :
virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
04:58:30.400+0000: 809: error : virQEMUCapsCacheLookupDefault:5577 :
invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on
this host 2022-01-10 04:58:30.408+0000: 807: error :
virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
04:58:30.415+0000: 808: error : virQEMUCapsCacheLookupDefault:5577 :
invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64'
on this host 2022-03-03 16:39:08.922+0000: 807: error :
qemuDomainDefValidate:5725 : unsupported configuration: Emulator
'/usr/bin/qemu-system-arm' does not support virt type 'kvm'

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: kvm libvirt qemu rasberrypi

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1963553

Title:
  Openstack Fails to Launch Instances "/usr/bin/qemu-system-arm' does
  not support virt type 'kvm; "

Status in OpenStack Compute (nova):
  New

Bug description:
  Environment - Openstack Victoria on Ubuntu 20.0.4 on Raspberry Pi 4B,
  1 controller, 2 compute, 1 storage nodes.

  Been troubleshooting an Raspberry Pi 4B Openstack setup. I have all my
  openstack components running and verified but when trying to launch an
  instance it errors out (seems to be from libvirt):

   Emulator '/usr/bin/qemu-system-arm' does not support virt type
  'kvm'\n", '\nDuring handling of the above exception, another exception
  occurred

  Per these instructions
  (https://docs.openstack.org/nova/victoria/configuration/config.html)
  as a fix I tried I've tried the following options in nova.conf in all
  variations with no difference in outcome (Restarted Libvirt and Nova-
  Computer service after each change as well)

  cpu_mode = (default)

  cpu_mode = host-passthrough

  virt_type = kvm

  virt_type = qemu

  I'm at a loss for what to do as to my knowledge the only way
  information is passed through to libvirt is through nova.conf, and
  appreciate any assistance.

  Controller Node /var/log/nova-conductor.log

      2022-03-03 16:39:13.462 1987 ERROR nova.scheduler.utils [req-de8dd1d0-3ff5-4495-ad8b-8af235b4d8c4 d20aa0616f264b39a2b72422d2d5d947 - - default default] [instance: c453aedc-08d5-4b5c-95c9-ddda1eab4514] Error from last host: compute2 (node compute2): ['Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2442, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 3766, in spawn\n self._create_guest_with_network(\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6758, in _create_guest_with_network\n
      self.cleanup_failed_start(\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in exit\n self.force_reraise()\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type, self.value, self.tb)\n', ' File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise\n
      raise value\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6727, in _create_guest_with_network\n guest = self._create_guest(\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 6655, in _create_guest\n guest = libvirt_guest.Guest.create(xml, self.host)\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 144, in create\n LOG.error('Error defining a guest with XML: %s',\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in exit\n self.force_reraise()\n', ' File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise\n six.reraise(self.type, self.value, self.tb)\n', ' File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise\n
      raise value\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py", line 141, in create\n guest = host.write_instance_config(xml)\n', ' File "/usr/lib/python3/dist-packages/nova/virt/libvirt/host.py", line 1144, in write_instance_config\n domain = self.get_connection().defineXML(xml)\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 193, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 151, in proxy_call\n rv = execute(f, *args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 132, in execute\n six.reraise(c, e, tb)\n', ' File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise\n
      raise value\n', ' File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 86, in tworker\n rv = meth(*args, **kwargs)\n', ' File "/usr/lib/python3/dist-packages/libvirt.py", line 4047, in defineXML\n if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)\n', "libvirt.libvirtError: unsupported configuration: Emulator '/usr/bin/qemu-system-arm' does not support virt type 'kvm'\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2268, in _do_build_and_run_instance\n
      self._build_and_run_instance(context, instance, image,\n', ' File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2538, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance c453aedc-08d5-4b5c-95c9-ddda1eab4514 was re-scheduled: unsupported configuration: Emulator '/usr/bin/qemu-system-arm' does not support virt type 'kvm'\n"] 2022-03-03 16:39:13.466 1987 WARNING nova.scheduler.utils [req-de8dd1d0-3ff5-4495-ad8b-8af235b4d8c4 d20aa0616f264b39a2b72422d2d5d947 - - default default] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c453aedc-08d5-4b5c-95c9-ddda1eab4514.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c453aedc-08d5-4b5c-95c9-ddda1eab4514.

  Compute Node libvirtd debug log

      2022-01-10 04:58:27.670+0000: 809: error : virCPUGetHost:439 :
  this function is not supported by the connection driver: cannot detect
  host CPU model for armv7l architecture 2022-01-10 04:58:27.670+0000:
  809: error : virCPUGetHost:439 : this function is not supported by the
  connection driver: cannot detect host CPU model for armv7l
  architecture 2022-01-10 04:58:30.362+0000: 809: error :
  virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
  supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
  04:58:30.370+0000: 807: error : virQEMUCapsCacheLookupDefault:5577 :
  invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm'
  on this host 2022-01-10 04:58:30.379+0000: 808: error :
  virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
  supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
  04:58:30.387+0000: 806: error : virQEMUCapsCacheLookupDefault:5577 :
  invalid argument: KVM is not supported by '/usr/bin/qemu-system-
  aarch64' on this host 2022-01-10 04:58:30.393+0000: 810: error :
  virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
  supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
  04:58:30.400+0000: 809: error : virQEMUCapsCacheLookupDefault:5577 :
  invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm'
  on this host 2022-01-10 04:58:30.408+0000: 807: error :
  virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
  supported by '/usr/bin/qemu-system-arm' on this host 2022-01-10
  04:58:30.415+0000: 808: error : virQEMUCapsCacheLookupDefault:5577 :
  invalid argument: KVM is not supported by '/usr/bin/qemu-system-
  aarch64' on this host 2022-03-03 16:39:08.922+0000: 807: error :
  qemuDomainDefValidate:5725 : unsupported configuration: Emulator
  '/usr/bin/qemu-system-arm' does not support virt type 'kvm'

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1963553/+subscriptions
Follow ups

[Bug 1963553] Re: Openstack Fails to Launch Instances "/usr/bin/qemu-system-arm' does not support virt type 'kvm; "
From: Sylvain Bauza, 2022-03-08