yahoo-eng-team team mailing list archive

[Sergio Rabellino <2043985@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Launching a libvirt vm with a UEFI boot with PCI cards (e.g. PCI passthrough of GPU cards) results in a black screen and no boot sequence at all.
This is due to limitations in UEFI Bios boot code (OVMF) which limits to 32GB the max memory mapping, so using GPUs that owns more memory than 32GB (summing up all the GPUs attached) the UEFI boot will fail to start at all.

Following the documentation, latest OVMF has a special flag that can be
used to ask to UEFI to allow more than 32GB mem alloc to PCI memory map
( https://edk2.groups.io/g/discuss/topic/59340711 ) which can be
specified into the domain XML file for the VM.

The attached patches are a workaround to allow using UEFI with GPUs
(tested on a Rocky 9.x with 4 nVIDIA V100 SXM2 cards, with a total of
128GB of pci memory map). The patch tries to sum-up from the host the
needed memory aperture and tries to understand if the current OVMF
release has the needed options (this should be enhanced further).

HTH

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: gpu libvirt uefi

** Patch added: "GIT diff result from latest nova git repository master"
   https://bugs.launchpad.net/bugs/2043985/+attachment/5721200/+files/diff_latest_nova.diff

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2043985

Title:
  Cannot start a UEFI vm when pci memory mapped exceed 32GB

Status in OpenStack Compute (nova):
  New

Bug description:
  Launching a libvirt vm with a UEFI boot with PCI cards (e.g. PCI passthrough of GPU cards) results in a black screen and no boot sequence at all.
  This is due to limitations in UEFI Bios boot code (OVMF) which limits to 32GB the max memory mapping, so using GPUs that owns more memory than 32GB (summing up all the GPUs attached) the UEFI boot will fail to start at all.

  Following the documentation, latest OVMF has a special flag that can
  be used to ask to UEFI to allow more than 32GB mem alloc to PCI memory
  map ( https://edk2.groups.io/g/discuss/topic/59340711 ) which can be
  specified into the domain XML file for the VM.

  The attached patches are a workaround to allow using UEFI with GPUs
  (tested on a Rocky 9.x with 4 nVIDIA V100 SXM2 cards, with a total of
  128GB of pci memory map). The patch tries to sum-up from the host the
  needed memory aperture and tries to understand if the current OVMF
  release has the needed options (this should be enhanced further).

  HTH

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2043985/+subscriptions