yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2040449] [NEW] Instance with memory encryption enabled can't be launched when [libvirt] cpu_mode is custom

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Takashi Kajinami <2040449@xxxxxxxxxxxxxxxxxx>
Date: Wed, 25 Oct 2023 06:50:46 -0000
Reply-to: Bug 2040449 <2040449@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Description
===========
When a user tries to launch an instance with memory encryption enabled, the instance always becomes error state if the nova-compute has [libvirt] cpu_mode = custom.

Steps to reproduce
==================
1. Set the following options in nova.conf and restart nova-compute

[libvirt]
cpu_mode = custom
cpu_models = EPYC

2. Prepare a flavor with memory encryption enabled

$ openstack flavor show m1.small-enc -f yaml
OS-FLV-DISABLED:disabled: false
OS-FLV-EXT-DATA:ephemeral: 0
access_project_ids: null
description: null
disk: 20
id: ee97652f-8948-4cdd-a5cd-71411cf9c8e4
name: m1.small-enc
os-flavor-access:is_public: true
properties:
  hw:mem_encryption: 'true'
ram: 2048
rxtx_factor: 1.0
swap: 0
vcpus: 1

3. Create an image with hw_firmware_type property set to 'uefi'

$ openstack image show cirros-uefi -f yaml
checksum: c8fc807773e5354afe61636071771906
container_format: bare
created_at: '2023-10-25T02:46:57Z'
disk_format: qcow2
file: /v2/images/d6353363-f580-464c-9909-93212298a58a/file
id: d6353363-f580-464c-9909-93212298a58a
min_disk: 0
min_ram: 0
name: cirros-uefi
owner: 5a2803c4cdb1412fa1e83738d7821904
properties:
  hw_disk_bus: scsi
  hw_firmware_type: uefi
  hw_scsi_model: virtio-scsi
  os_hash_algo: sha512
  os_hash_value: 1103b92ce8ad966e41235a4de260deb791ff571670c0342666c8582fbb9caefe6af07ebb11d34f44f8414b609b29c1bdf1d72ffa6faa39c88e8721d09847952b
  os_hidden: false
  owner_specified.openstack.md5: ''
  owner_specified.openstack.object: images/cirros-uefi
  owner_specified.openstack.sha256: ''
  stores: fs
protected: false
schema: /v2/schemas/image
size: 21430272
status: active
tags: []
updated_at: '2023-10-25T06:00:15Z'
virtual_size: 117440512
visibility: public

4. launch an instance using the flavr and the image
$ openstack server create --image cirros-uefi --flavor m1.small-enc --network private cirros-enc

Expected result
===============
The instance becomes active state

Actual result
=============
Instance becomes error state. The following traceback is found in nova-compute.log

```
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [None req-104288bc-7bf5-4bcd-a728-cd85ac72416f 69d6ccfef7e240398970c80f0be8ccf7 5a2803c4cdb1412fa1e83738d7821904 - - default default] [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542] Failed to build and run instance: nova.exception.FlavorImageConflict: Memory encryption requested by hw:mem_encryption extra spec in m1.small-enc flavor but image None doesn't have 'hw_firmware_type' property set to 'uefi' or volume-backed instance was requested
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542] Traceback (most recent call last):
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2615, in _build_and_run_instance
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     self.driver.spawn(context, instance, image_meta,
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4413, in spawn
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     xml = self._get_guest_xml(context, instance, network_info,
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7565, in _get_guest_xml
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     conf = self._get_guest_config(instance, network_info, image_meta,
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7045, in _get_guest_config
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     guest.cpu = self._get_guest_cpu_config(
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 5451, in _get_guest_cpu_config
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     cpu = self._get_guest_cpu_model_config(flavor, arch)
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 5398, in _get_guest_cpu_model_config
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     flags = libvirt_utils.get_flags_by_flavor_specs(flavor)
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/utils.py", line 658, in get_flags_by_flavor_specs
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     resource_request = scheduler_utils.ResourceRequest.from_request_spec(
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 187, in from_request_spec
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     res_req._translate_memory_encryption(request_spec.flavor, image)
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 297, in _translate_memory_encryption
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     if not hardware.get_mem_encryption_constraint(flavor, image):
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/hardware.py", line 1197, in get_mem_encryption_constraint
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     _check_mem_encryption_uses_uefi_image(requesters, image_meta)
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/hardware.py", line 1243, in _check_mem_encryption_uses_uefi_image
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     raise exception.FlavorImageConflict(emsg % data)
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542] nova.exception.FlavorImageConflict: Memory encryption requested by hw:mem_encryption extra spec in m1.small-enc flavor but image None doesn't have 'hw_firmware_type' property set to 'uefi' or volume-backed instance was requested
2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]
```

Environment
===========
The issue was found in a deployment using Ubuntu 22.04 + UCA Bobcat.
Nova packages are all 3:28.0.0-0ubuntu1~cloud0

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2040449

Title:
  Instance with memory encryption enabled can't be launched when
  [libvirt] cpu_mode is custom

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  When a user tries to launch an instance with memory encryption enabled, the instance always becomes error state if the nova-compute has [libvirt] cpu_mode = custom.

  Steps to reproduce
  ==================
  1. Set the following options in nova.conf and restart nova-compute

  [libvirt]
  cpu_mode = custom
  cpu_models = EPYC

  2. Prepare a flavor with memory encryption enabled

  $ openstack flavor show m1.small-enc -f yaml
  OS-FLV-DISABLED:disabled: false
  OS-FLV-EXT-DATA:ephemeral: 0
  access_project_ids: null
  description: null
  disk: 20
  id: ee97652f-8948-4cdd-a5cd-71411cf9c8e4
  name: m1.small-enc
  os-flavor-access:is_public: true
  properties:
    hw:mem_encryption: 'true'
  ram: 2048
  rxtx_factor: 1.0
  swap: 0
  vcpus: 1

  3. Create an image with hw_firmware_type property set to 'uefi'

  $ openstack image show cirros-uefi -f yaml
  checksum: c8fc807773e5354afe61636071771906
  container_format: bare
  created_at: '2023-10-25T02:46:57Z'
  disk_format: qcow2
  file: /v2/images/d6353363-f580-464c-9909-93212298a58a/file
  id: d6353363-f580-464c-9909-93212298a58a
  min_disk: 0
  min_ram: 0
  name: cirros-uefi
  owner: 5a2803c4cdb1412fa1e83738d7821904
  properties:
    hw_disk_bus: scsi
    hw_firmware_type: uefi
    hw_scsi_model: virtio-scsi
    os_hash_algo: sha512
    os_hash_value: 1103b92ce8ad966e41235a4de260deb791ff571670c0342666c8582fbb9caefe6af07ebb11d34f44f8414b609b29c1bdf1d72ffa6faa39c88e8721d09847952b
    os_hidden: false
    owner_specified.openstack.md5: ''
    owner_specified.openstack.object: images/cirros-uefi
    owner_specified.openstack.sha256: ''
    stores: fs
  protected: false
  schema: /v2/schemas/image
  size: 21430272
  status: active
  tags: []
  updated_at: '2023-10-25T06:00:15Z'
  virtual_size: 117440512
  visibility: public

  4. launch an instance using the flavr and the image
  $ openstack server create --image cirros-uefi --flavor m1.small-enc --network private cirros-enc

  Expected result
  ===============
  The instance becomes active state

  Actual result
  =============
  Instance becomes error state. The following traceback is found in nova-compute.log

  ```
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [None req-104288bc-7bf5-4bcd-a728-cd85ac72416f 69d6ccfef7e240398970c80f0be8ccf7 5a2803c4cdb1412fa1e83738d7821904 - - default default] [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542] Failed to build and run instance: nova.exception.FlavorImageConflict: Memory encryption requested by hw:mem_encryption extra spec in m1.small-enc flavor but image None doesn't have 'hw_firmware_type' property set to 'uefi' or volume-backed instance was requested
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542] Traceback (most recent call last):
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line 2615, in _build_and_run_instance
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     self.driver.spawn(context, instance, image_meta,
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 4413, in spawn
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     xml = self._get_guest_xml(context, instance, network_info,
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7565, in _get_guest_xml
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     conf = self._get_guest_config(instance, network_info, image_meta,
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 7045, in _get_guest_config
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     guest.cpu = self._get_guest_cpu_config(
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 5451, in _get_guest_cpu_config
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     cpu = self._get_guest_cpu_model_config(flavor, arch)
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py", line 5398, in _get_guest_cpu_model_config
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     flags = libvirt_utils.get_flags_by_flavor_specs(flavor)
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/libvirt/utils.py", line 658, in get_flags_by_flavor_specs
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     resource_request = scheduler_utils.ResourceRequest.from_request_spec(
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 187, in from_request_spec
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     res_req._translate_memory_encryption(request_spec.flavor, image)
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/scheduler/utils.py", line 297, in _translate_memory_encryption
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     if not hardware.get_mem_encryption_constraint(flavor, image):
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/hardware.py", line 1197, in get_mem_encryption_constraint
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     _check_mem_encryption_uses_uefi_image(requesters, image_meta)
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]   File "/usr/lib/python3/dist-packages/nova/virt/hardware.py", line 1243, in _check_mem_encryption_uses_uefi_image
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]     raise exception.FlavorImageConflict(emsg % data)
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542] nova.exception.FlavorImageConflict: Memory encryption requested by hw:mem_encryption extra spec in m1.small-enc flavor but image None doesn't have 'hw_firmware_type' property set to 'uefi' or volume-backed instance was requested
  2023-10-25 06:33:20.674 38337 ERROR nova.compute.manager [instance: 000b22bc-6b28-4adb-a3af-44b1f090c542]
  ```

  Environment
  ===========
  The issue was found in a deployment using Ubuntu 22.04 + UCA Bobcat.
  Nova packages are all 3:28.0.0-0ubuntu1~cloud0

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2040449/+subscriptions