yahoo-eng-team team mailing list archive

[Marcin Juszkiewicz <1868464@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

During queens cycle we set hw_firmware_type to be uefi for aarch64
architecture (as it is the only sane way to boot an instance there).

Since then images do not need to have 'hw_firmware_type' property set to
get instances working.

The problem starts when there are issues starting instance for other
reasons:

│2020-03-21 19:35:04.451 6 ERROR nova.compute.manager [req-9a494e07-cfd7
-49ca-87d2-bbe3ac86bc2c 9f732df1c71f4788b1b834c07511bc53
acc9e58c6c514ac1af7d3c28e9d690ee - default default] [instance:
09554b80-bb49-47bb-bbd3-36d651ab4655] Instance failed to spawn:
libvirt.libvirtError: Requested operation is not valid: cannot undefine
domain with nvram

I went through code and found out that when machine starts on aarch64
then hw_firmware_type=UEFI is set in code by default (since queens). It
is not stored in image metadata at all (when image is checked with
'image show IMAGENAME').

But when I start such image and it fails to start then it looks like
nova does not set flag to remove nvram. I looked at libvirt/driver.py
and it looks like it is assumed that instance/image will have
hw_firmware_type=UEFI set ;(

When I added hw_firmware_type=uefi to the image the problem was gone.
But it is not proper solution.

Something like this probably needs to be done in all places where it is
checked:

diff --git nova/virt/libvirt/driver.py nova/virt/libvirt/driver.py
index 45af21c3bb..0f293bbea8 100644
--- nova/virt/libvirt/driver.py
+++ nova/virt/libvirt/driver.py
@@ -1270,11 +1270,7 @@ class LibvirtDriver(driver.ComputeDriver):
         try:
             guest = self._host.get_guest(instance)
             try:
-                hw_firmware_type = instance.image_meta.properties.get(
-                    'hw_firmware_type')
-                support_uefi = (self._has_uefi_support() and
-                                hw_firmware_type == fields.FirmwareType.UEFI)
-                guest.delete_configuration(support_uefi)
+                guest.delete_configuration(self._has_uefi_support())
             except libvirt.libvirtError as e:
                 with excutils.save_and_reraise_exception() as ctxt:
                     errcode = e.get_error_code()

Or maybe set NVRAM flag each time as 'hw_firmware_type' can not be
assumed to be set.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1868464

Title:
  Nova assumes hw_firmware_type=uefi being set on UEFI instances

Status in OpenStack Compute (nova):
  New

Bug description:
  During queens cycle we set hw_firmware_type to be uefi for aarch64
  architecture (as it is the only sane way to boot an instance there).

  Since then images do not need to have 'hw_firmware_type' property set
  to get instances working.

  The problem starts when there are issues starting instance for other
  reasons:

  │2020-03-21 19:35:04.451 6 ERROR nova.compute.manager [req-
  9a494e07-cfd7-49ca-87d2-bbe3ac86bc2c 9f732df1c71f4788b1b834c07511bc53
  acc9e58c6c514ac1af7d3c28e9d690ee - default default] [instance:
  09554b80-bb49-47bb-bbd3-36d651ab4655] Instance failed to spawn:
  libvirt.libvirtError: Requested operation is not valid: cannot
  undefine domain with nvram

  I went through code and found out that when machine starts on aarch64
  then hw_firmware_type=UEFI is set in code by default (since queens).
  It is not stored in image metadata at all (when image is checked with
  'image show IMAGENAME').

  But when I start such image and it fails to start then it looks like
  nova does not set flag to remove nvram. I looked at libvirt/driver.py
  and it looks like it is assumed that instance/image will have
  hw_firmware_type=UEFI set ;(

  When I added hw_firmware_type=uefi to the image the problem was gone.
  But it is not proper solution.

  Something like this probably needs to be done in all places where it
  is checked:

  diff --git nova/virt/libvirt/driver.py nova/virt/libvirt/driver.py
  index 45af21c3bb..0f293bbea8 100644
  --- nova/virt/libvirt/driver.py
  +++ nova/virt/libvirt/driver.py
  @@ -1270,11 +1270,7 @@ class LibvirtDriver(driver.ComputeDriver):
           try:
               guest = self._host.get_guest(instance)
               try:
  -                hw_firmware_type = instance.image_meta.properties.get(
  -                    'hw_firmware_type')
  -                support_uefi = (self._has_uefi_support() and
  -                                hw_firmware_type == fields.FirmwareType.UEFI)
  -                guest.delete_configuration(support_uefi)
  +                guest.delete_configuration(self._has_uefi_support())
               except libvirt.libvirtError as e:
                   with excutils.save_and_reraise_exception() as ctxt:
                       errcode = e.get_error_code()

  Or maybe set NVRAM flag each time as 'hw_firmware_type' can not be
  assumed to be set.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1868464/+subscriptions