yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1867075] Re: Arm64: Instance with Configure Drive attach volume failed failed

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Elod Illes <1867075@xxxxxxxxxxxxxxxxxx>
Date: Tue, 24 Nov 2020 13:43:45 -0000
Reply-to: Bug 1867075 <1867075@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova/stein
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1867075

Title:
  Arm64: Instance with Configure Drive attach volume failed failed

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) queens series:
  Fix Committed
Status in OpenStack Compute (nova) rocky series:
  Fix Committed
Status in OpenStack Compute (nova) stein series:
  Fix Released
Status in OpenStack Compute (nova) train series:
  Fix Committed

Bug description:
  Arm64.

  Image: cirros-0.5.1
  hw_cdrom_bus='scsi', hw_disk_bus='scsi', hw_machine_type='virt', hw_rng_model='virtio', hw_scsi_model='virtio-scsi', os_command_line=''console=ttyAMA0''

  Boot a vm.
  Create a volume: openstack volume create --size 1 test

  Attach:
  openstack server add volume cirros-test test

  Error:
  DEBUG nova.virt.libvirt.guest [None req-8dfbf677-50bb-42be-869f-52c9ac638d59 admin admin] attach device xml: 
  <disk type="block" device="disk">
    <driver name="qemu" type="raw" cache="none" io="native"/>                                                                                                                                                                                   
    <source dev="/dev/sdc"/>
    <target dev="sdc" bus="scsi"/>                                                                                                                                                                                                              
    <serial>b9abb789-1c55-4210-ab5c-78b0e3619405</serial>                                                                                                                                                                                       
    <address type="drive" controller="0" unit="1"/>                                                                                                                                                                                           </disk>

  ror: Requested operation is not valid: Domain already contains a disk with that address
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6] Traceback (most recent call last):
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/opt/stack/nova/nova/virt/block_device.py", line 599, in _volume_attach
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     device_type=self['device_type'], encryption=encryption)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1731, in attach_volume
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     conf = self._get_volume_config(connection_info, disk_info)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     self.force_reraise()
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     six.reraise(self.type_, self.value, self.tb)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     raise value
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1731, in attach_volume
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     conf = self._get_volume_config(connection_info, disk_info)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 293, in attach_device
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     self._domain.attachDeviceFlags(device_xml, flags=flags)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 190, in doit
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 148, in proxy_call
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     rv = execute(f, *args, **kwargs)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 129, in execute
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     six.reraise(c, e, tb)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     raise value
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/eventlet/tpool.py", line 83, in tworker
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     rv = meth(*args, **kwargs)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]   File "/usr/local/lib/python3.6/dist-packages/libvirt.py", line 593, in attachDeviceFlags
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]     if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self)
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6] libvirt.libvirtError: Requested operation is not valid: Domain already contains a disk with that address
  ERROR nova.virt.block_device [instance: 22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6]

  <domain type='qemu' id='327'>
    <name>instance-000000f8</name>
    <uuid>22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6</uuid>
    <metadata>
      <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0";>
        <nova:package version="20.1.0"/>
        <nova:name>cirros-test</nova:name>
        <nova:creationTime>2020-03-12 01:35:31</nova:creationTime>
        <nova:flavor name="m1.tiny">
          <nova:memory>512</nova:memory>
          <nova:disk>1</nova:disk>
          <nova:swap>0</nova:swap>
          <nova:ephemeral>0</nova:ephemeral>
          <nova:vcpus>1</nova:vcpus>
        </nova:flavor>
        <nova:owner>
          <nova:user uuid="4bb28beb8465489f96441dfda663e4bd">admin</nova:user>
          <nova:project uuid="c4838cf5ab504adc9f61f717532ededd">admin</nova:project>
        </nova:owner>
        <nova:root type="image" uuid="3937ae70-18b5-4bbd-989b-af123ce2af4a"/>
      </nova:instance>
    </metadata>
    <memory unit='KiB'>524288</memory>
    <currentMemory unit='KiB'>524288</currentMemory>
    <vcpu placement='static'>1</vcpu>
    <cputune>
      <shares>1024</shares>
    </cputune>
    <resource>
      <partition>/machine</partition>
    </resource>
    <os>
      <type arch='aarch64' machine='virt-2.11'>hvm</type>
      <loader readonly='yes' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader>
      <nvram>/var/lib/libvirt/qemu/nvram/instance-000000f8_VARS.fd</nvram>
      <boot dev='hd'/>
    </os>
    <features>
      <acpi/>
      <apic/>
      <gic version='2'/>
    </features>
    <cpu mode='custom' match='exact' check='none'>
      <model fallback='forbid'>cortex-a57</model>
      <topology sockets='1' cores='1' threads='1'/>
    </cpu>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <emulator>/usr/bin/qemu-system-aarch64</emulator>
      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2' cache='none'/>
        <source file='/opt/stack/data/nova/instances/22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6/disk'/>
        <backingStore type='file' index='1'>
          <format type='raw'/>
          <source file='/opt/stack/data/nova/instances/_base/9707f79a24b3945e5b508b0699a5eb97b15279cc'/>
          <backingStore/>
        </backingStore>
        <target dev='sda' bus='scsi'/>
        <alias name='scsi0-0-0-0'/>
        <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      </disk>
      <disk type='file' device='cdrom'>
        <driver name='qemu' type='raw' cache='none'/>
        <source file='/opt/stack/data/nova/instances/22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6/disk.config'/>
        <backingStore/>
        <target dev='sdb' bus='scsi'/>
        <readonly/>
        <alias name='scsi0-0-0-1'/>
        <address type='drive' controller='0' bus='0' target='0' unit='1'/>
      </disk>
      <controller type='scsi' index='0' model='virtio-scsi'>
        <alias name='scsi0'/>
        <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </controller>
      <controller type='pci' index='0' model='pcie-root'>
        <alias name='pcie.0'/>
      </controller>
      <controller type='pci' index='1' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='1' port='0x8'/>
        <alias name='pci.1'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
      </controller>
      <controller type='pci' index='2' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='2' port='0x9'/>
        <alias name='pci.2'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
      </controller>
      <controller type='pci' index='3' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='3' port='0xa'/>
        <alias name='pci.3'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
      </controller>
      <controller type='pci' index='4' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='4' port='0xb'/>
        <alias name='pci.4'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
      </controller>
      <controller type='pci' index='5' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='5' port='0xc'/>
        <alias name='pci.5'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
      </controller>
      <controller type='pci' index='6' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='6' port='0xd'/>
        <alias name='pci.6'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
      </controller>
      <controller type='pci' index='7' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='7' port='0xe'/>
        <alias name='pci.7'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
      </controller>
      <controller type='pci' index='8' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='8' port='0xf'/>
        <alias name='pci.8'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
      </controller>
      <controller type='pci' index='9' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='9' port='0x10'/>
        <alias name='pci.9'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
      </controller>
      <controller type='pci' index='10' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='10' port='0x11'/>
        <alias name='pci.10'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
      </controller>
      <controller type='pci' index='11' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='11' port='0x12'/>
        <alias name='pci.11'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
      </controller>
      <controller type='pci' index='12' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='12' port='0x13'/>
        <alias name='pci.12'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
      </controller>
      <controller type='pci' index='13' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='13' port='0x14'/>
        <alias name='pci.13'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
      </controller>
      <controller type='pci' index='14' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='14' port='0x15'/>
        <alias name='pci.14'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
      </controller>
      <controller type='pci' index='15' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='15' port='0x16'/>
        <alias name='pci.15'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
      </controller>
      <controller type='pci' index='16' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='16' port='0x17'/>
        <alias name='pci.16'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
      </controller>
      <controller type='pci' index='17' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='17' port='0x18'/>
        <alias name='pci.17'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
      </controller>
      <controller type='pci' index='18' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='18' port='0x19'/>
        <alias name='pci.18'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
      </controller>
      <controller type='pci' index='19' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='19' port='0x1a'/>
        <alias name='pci.19'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
      </controller>
      <controller type='pci' index='20' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='20' port='0x1b'/>
        <alias name='pci.20'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
      </controller>
      <controller type='pci' index='21' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='21' port='0x1c'/>
        <alias name='pci.21'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
      </controller>
      <controller type='pci' index='22' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='22' port='0x1d'/>
        <alias name='pci.22'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x5'/>
      </controller>
      <controller type='pci' index='23' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='23' port='0x1e'/>
        <alias name='pci.23'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x6'/>
      </controller>
      <controller type='pci' index='24' model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='24' port='0x1f'/>
        <alias name='pci.24'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x7'/>
      </controller>
      <interface type='bridge'>
        <mac address='fa:16:3e:11:3e:c3'/>
        <source bridge='br-int'/>
        <virtualport type='openvswitch'>
          <parameters interfaceid='2e718998-be15-4b02-ac8a-495b940ad6d0'/>
        </virtualport>
        <target dev='tap2e718998-be'/>
        <model type='virtio'/>
        <driver name='qemu'/>
        <mtu size='1450'/>
        <alias name='net0'/>
        <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </interface>
      <serial type='pty'>
        <source path='/dev/pts/9'/>
        <log file='/opt/stack/data/nova/instances/22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6/console.log' append='off'/>
        <target type='system-serial' port='0'>
          <model name='pl011'/>
        </target>
        <alias name='serial0'/>
      </serial>
      <console type='pty' tty='/dev/pts/9'>
        <source path='/dev/pts/9'/>
        <log file='/opt/stack/data/nova/instances/22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6/console.log' append='off'/>
        <target type='serial' port='0'/>
        <alias name='serial0'/>
      </console>
      <memballoon model='virtio'>
        <stats period='10'/>
        <alias name='balloon0'/>
        <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </memballoon>
      <rng model='virtio'>
        <backend model='random'>/dev/urandom</backend>
        <alias name='rng0'/>
        <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
      </rng>
    </devices>
    <seclabel type='dynamic' model='apparmor' relabel='yes'>
      <label>libvirt-22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6</label>
      <imagelabel>libvirt-22bdc0a6-1c0c-43fa-8c64-66735b6a6cb6</imagelabel>
    </seclabel>
    <seclabel type='dynamic' model='dac' relabel='yes'>
      <label>+64055:+123</label>
      <imagelabel>+64055:+123</imagelabel>
    </seclabel>
  </domain>

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1867075/+subscriptions
References

[Bug 1867075] [NEW] Arm64: Instance with Configure Drive attach volume failed failed
From: Kevin Zhao, 2020-03-12