← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2100588] Re: Incorrect image ID written to libvirt metadata when unshelving ephemeral instances

 

Reviewed:  https://review.opendev.org/c/openstack/nova/+/942973
Committed: https://opendev.org/openstack/nova/commit/4c64f84058560be95af9ae6e71625e52986d32f9
Submitter: "Zuul (22348)"
Branch:    master

commit 4c64f84058560be95af9ae6e71625e52986d32f9
Author: Callum Dickinson <callum.dickinson@xxxxxxxxxxxxxxxx>
Date:   Fri Feb 28 16:26:43 2025 +1300

    Fix image ID in libvirt metadata when unshelving
    
    Nova adds the temporary shelved image ID to libvirt metadata
    when unshelving image-backed instances. This is corrected when
    the instance is cold restarted, resized or migrated but causes
    issues for other services such as Ceilometer which rely on this
    data being correct.
    
    This patch ensures the correct image ID is set in the libvirt
    domain metadata when image-backed instances are unshelved.
    
    Signed-off-by: Callum Dickinson <callum.dickinson@xxxxxxxxxxxxxxxx>
    Co-Authored-By: Jeremy Lamb <jeremy.lamb@xxxxxxxxxxxxxxxx>
    Closes-Bug: #2100588
    Change-Id: Ifd9f092299912606931848b2b25b4be6b36effac


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2100588

Title:
  Incorrect image ID written to libvirt metadata when unshelving
  ephemeral instances

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Description
  ===========

  When an ephemeral instance (instance booted from image) is shelved,
  Nova writes the root volume of the instance to a temporary Glance
  image to make sure it persists while the instance is shelved. Upon
  unshelve the volume is written back out to disk, and the temporary
  image is removed.

  Upon unshelving the instance, when the libvirt domain metadata for the
  instance is generated the temporary image's ID is written to the
  metadata, instead of the ID of the original image the instance is
  based on like it should.

  This is corrected when the instance is cold restarted, resized or cold
  migrated but the incorrect image ID can persist if the state of the
  instance does not change following the unshelve, which causes issues
  for other services such as Ceilometer which rely on this data being
  correct.

  Steps to reproduce
  ==================

  1. Launch a new ephemeral instance, booted from image.

  2. Confirm the correct image ID of the active ephemeral instance:

  $ openstack server show <instance>

  3. Check the libvirt domain XML (the image ID should be correct):

  $ virsh dumpxml instance-xxxxxxxx

  4. Shelve the instance:

  $ openstack server shelve <instance>

  5. Unshelve the instance:

  $ openstack server unshelve <instance>

  6. Check the libvirt domain XML again (the image ID will be
  incorrect):

  $ virsh dumpxml instance-xxxxxxxx

  Expected result
  ===============

  Given the following list of images (taken while the instance 'test2'
  is shelved):

  $ openstack image list
  +--------------------------------------+--------------------------------------+--------+
  | ID                                   | Name                                 | Status |
  +--------------------------------------+--------------------------------------+--------+
  | 00000000-0000-0000-0000-000000000001 | alt_tempest_cirros-0.3.4-x86_64      | active |
  | 4c1e8dbe-7ef5-44be-8b3c-e36f72d93589 | cirros-0.6.2-x86_64                  | active |
  | a734e8cf-376d-4d00-a51e-b57995a4fc24 | octavia_image                        | active |
  | 00000000-0000-0000-0000-000000000000 | tempest_cirros-0.3.4-x86_64          | active |
  | f8e04922-c715-4ffe-82ef-cddace05f331 | test2-shelved                        | active |
  | e69daa32-b950-40d6-bc5e-599e73f5c838 | trove-victoria-ubuntu-focal-20240514 | active |
  +--------------------------------------+--------------------------------------+--------+

  libvirt domain metadata before unshelve:

  $ sudo virsh dumpxml 7a8af73e-9aa2-4f5d-bb20-7f5623adcd58 | xpath -q -e /domain/metadata
  <metadata>
      <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0";>
        <nova:package version="17.0.14" />
        <nova:name>test2</nova:name>
        <nova:creationTime>2025-02-27 01:26:17</nova:creationTime>
        <nova:flavor name="c1.c1r05">
          <nova:memory>512</nova:memory>
          <nova:disk>5</nova:disk>
          <nova:swap>0</nova:swap>
          <nova:ephemeral>0</nova:ephemeral>
          <nova:vcpus>1</nova:vcpus>
        </nova:flavor>
        <nova:owner>
          <nova:user uuid="a55b109e1508472cb4466bc84786680e">jeremy</nova:user>
          <nova:project uuid="ab5d28cb6e2c4b7d89c8e0f96ac8604e">test1</nova:project>
        </nova:owner>
        <nova:root type="image" uuid="4c1e8dbe-7ef5-44be-8b3c-e36f72d93589" /> <!-- cirros-0.6.2-x86_64 -->
      </nova:instance>
    </metadata>

  After unshelve, the image ID should stay the same.

  Actual result
  =============

  After unshelve:

  $ sudo virsh dumpxml 7a8af73e-9aa2-4f5d-bb20-7f5623adcd58 | xpath -q -e /domain/metadata
  <metadata>
      <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0";>
        <nova:package version="17.0.14" />
        <nova:name>test2</nova:name>
        <nova:creationTime>2025-02-27 01:17:00</nova:creationTime>
        <nova:flavor name="c1.c1r05">
          <nova:memory>512</nova:memory>
          <nova:disk>5</nova:disk>
          <nova:swap>0</nova:swap>
          <nova:ephemeral>0</nova:ephemeral>
          <nova:vcpus>1</nova:vcpus>
        </nova:flavor>
        <nova:owner>
          <nova:user uuid="a55b109e1508472cb4466bc84786680e">jeremy</nova:user>
          <nova:project uuid="ab5d28cb6e2c4b7d89c8e0f96ac8604e">test1</nova:project>
        </nova:owner>
        <nova:root type="image" uuid="f8e04922-c715-4ffe-82ef-cddace05f331" /> <!-- test2-shelved -->
      </nova:instance>
    </metadata>

  Environment
  ===========

  We have experienced it on Nova Queens through to Train, but looking at
  the implementation all Nova versions up to master are affected.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2100588/+subscriptions



References