yahoo-eng-team team mailing list archive

[OpenStack Infra <2055411@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/nova/+/910627
Committed: https://opendev.org/openstack/nova/commit/6a3ca95a36f4cae7a5b73b9f5663ef2c605f8bbb
Submitter: "Zuul (22348)"
Branch:    master

commit 6a3ca95a36f4cae7a5b73b9f5663ef2c605f8bbb
Author: Fabian Wiesel <fabian.wiesel@xxxxxxx>
Date:   Thu Mar 3 13:04:22 2022 +0100

    Vmware: Remove uuid parameter from get_vmdk_info call
    
    We changed the code to ignore the file-name as
    - a vmotion will result in renaming of the files
    - booting from a volume names the volume by its uuid,
    both breaking the heuristic to detect the root disk.
    
    We simply take the first hard disk in the default boot-order.
    If we boot from an ISO, it will be attached as a CD-ROM, not as a
    disk. Any snapshots would be taken from the first actual disk, so it
    still can be used as a method to install an OS to an empty disk.
    
    Disks for rescue operations are attached later in the default
    boot-order, but the boot order will be changed to allow booting from
    that disk.
    
    Closes-Bug: #2055411
    Change-Id: Ib3088cfce4f7a0b24f05d45e7830b011c4a39f42


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2055411

Title:
  Nova VMwareapi Resize of Volume Backed server fails

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Description
  ===========
  More specifically the following tempest test in master fails:
  tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON.test_rebuild_volume_backed_server

  
  Steps to reproduce
  ==================
  * Install Devstack from master
  * Run tempest test `tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON.test_rebuild_volume_backed_server`

  Expected result
  ===============
  The test succeeds.

  Actual result
  =============
  What happened instead of the expected result?
  How did the issue look like?

  Environment
  ===========
  1. Git 1858cf18b940b3636e54eb5aafaf4050bdd02939 (master). So essentially this:
   https://review.opendev.org/c/openstack/nova/+/909474
  As instance creation is impossible without that patch.

  2. Which hypervisor did you use? What's the version of that?

  vmwareapi (VSphere 7.0.3 & ESXi 7.0.3)

  2. Which storage type did you use?

  vmdk on NFS 4.1

  3. Which networking type did you use?

  networking-nsx-t (https://github.com/sapcc/networking-nsx-t)

  Logs & Configs
  ==============

  Can be found here: http://openstack-ci-
  logs.global.cloud.sap/openstack/nova/1858cf18b940b3636e54eb5aafaf4050bdd02939/index.html

  The critical exception for this bug report is (abbreviated and reformatted for clarity):
  ----
   req-7aa5ded6-ea97-4010-93c8-9e39389cbfe0 tempest-ServerActionsTestOtherA-839537081
  [  865.017199] env[58735]: ERROR nova.compute.manager [instance: b4d9131c-fc91-4fd4-813b-13b4bdfe1647] 
  Traceback (most recent call last):
    File "/opt/stack/nova/nova/compute/manager.py", line 10856, in _error_out_instance_on_exception
      yield
    File "/opt/stack/nova/nova/compute/manager.py", line 6096, in _resize_instance
      disk_info = self.driver.migrate_disk_and_power_off(
    File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 263, in migrate_disk_and_power_off
      return self._vmops.migrate_disk_and_power_off(context, instance,
    File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1467, in migrate_disk_and_power_off
      self._resize_disk(instance, vm_ref, vmdk, flavor)
    File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1398, in _resize_disk
      self._volumeops.detach_disk_from_vm(vm_ref, instance, vmdk.device)
    File "/opt/stack/nova/nova/virt/vmwareapi/volumeops.py", line 121, in detach_disk_from_vm
      disk_key = device.key
  AttributeError: 'NoneType' object has no attribute 'key'

  ---

  
  The bug is actually in the function `nova.virt.vmwareapi.vm_util.get_vmdk_info` here:
  https://opendev.org/openstack/nova/src/branch/master/nova/virt/vmwareapi/vm_util.py#L690

  The code works with the assumption, that the root-disk is named as the instance.
  This assumption breaks in several cases, but most for this test-case, the root volume is actually a cinder volume.
  It will also break when the the disk gets migrated to another datastore, either through a live-migration with no shared storage, or simply automatically with SDRS..

  I have an alternative implementation here: https://github.com/sapcc/nova/blob/stable/xena-m3/nova/virt/vmwareapi/vm_util.py#L997-L1034
  I'll provide a bug fix from it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2055411/+subscriptions