yahoo-eng-team team mailing list archive

[OpenStack Infra <1757190@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/554667
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5f3cca205581d45d92714ce1a909d4394b7812ff
Submitter: Zuul
Branch:    master

commit 5f3cca205581d45d92714ce1a909d4394b7812ff
Author: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date:   Tue Mar 20 15:04:27 2018 -0400

    Preserve multiattach flag when refreshing connection_info
    
    When we attach a multiattach-capable volume, we do something
    dirty and stash a "multiattach" boolean flag in the
    BlockDeviceMapping.connection_info dict. This is used by the
    virt driver to determine how to connect the volume (for the
    libvirt driver, it sets the "shareable" element on the disk
    config xml).
    
    When resizing an instance, ComputeManager._finish_resize on
    the destination host refreshes the block device mapping list
    along with the connection_info for each BDM. Because of this,
    it would overwrite the BDM.connection_info along with the stashed
    "multiattach" flag which is later used to connect the volumes
    on the destination host via the virt driver.finish_migration
    method. This leads to failures with multiattach volumes because
    the disk config is wrong.
    
    To fix this, when refreshing BDM connection_info, we preserve
    the multiattach flag in the connection_info, similar to the
    serial (volume ID) and multipath_id.
    
    Interestingly enough, the nova-multiattach job does not fail
    the volume multiattach resize test with libvirt 1.3.1 and qemu
    2.5. This failure was only noticed once the nova-multiattach
    job was tested with libvirt 4.0.0 and qemu 2.11.1. So maybe there
    was something in the older package versions that masked this
    obvious bug in the nova code.
    
    Change-Id: Iaee13478212cc04e6d1a1249f33822369d94d41d
    Closes-Bug: #1757190


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1757190

Title:
  resize fails with volume multiattach using with libvirt 4.0.0 (and
  qemu 2.11.1): Failed to get shared "write" lock

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) queens series:
  In Progress

Bug description:
  Seeing this in a patch https://review.openstack.org/#/c/554317/ that
  runs the nova-multiattach job with the Queens Ubuntu Cloud Archive
  which has libvirt 4.0.0 and qemu 2.11.1:

  http://logs.openstack.org/17/554317/1/check/nova-
  multiattach/8e97832/logs/libvirt/qemu/instance-00000066.txt.gz

  2018-03-19T19:48:16.175548Z qemu-system-x86_64: -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1: Failed to get shared "write" lock
  Is another process using the image?

  
  http://logs.openstack.org/17/554317/1/check/nova-multiattach/8e97832/logs/screen-n-cpu.txt.gz?level=TRACE#_Mar_19_19_48_16_261051

  Mar 19 19:48:17.132940 ubuntu-xenial-inap-mtl01-0003062768 nova-compute[27735]: ERROR nova.compute.manager [None req-3a092a4b-7ae7-4f29-9f78-97bf1dc0d46d service nova] [instance: 0eed0237-245e-4a18-9e30-9e72accd36c6] Setting instance vm_state to ERROR: libvirtError: internal error: process exited while connecting to monitor: 2018-03-19T19:48:16.147136Z qemu-system-x86_64: -drive file=/dev/sdb,format=raw,if=none,id=drive-virtio-disk1,serial=652600d5-f6dc-4089-ba95-d71d7640cafa,cache=none,aio=native: 'serial' is deprecated, please use the corresponding option of '-device' instead
  Mar 19 19:48:17.133724 ubuntu-xenial-inap-mtl01-0003062768 nova-compute[27735]: 2018-03-19T19:48:16.155115Z qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
  Mar 19 19:48:17.134022 ubuntu-xenial-inap-mtl01-0003062768 nova-compute[27735]: 2018-03-19T19:48:16.175548Z qemu-system-x86_64: -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1: Failed to get shared "write" lock

  That last error likely means the 'shareable' element isn't in the disk
  config xml, and it's not:

  <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/sdb'/>
      <target dev='vdb' bus='virtio'/>
      <serial>652600d5-f6dc-4089-ba95-d71d7640cafa</serial>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
  </disk>

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1757190/+subscriptions