yahoo-eng-team team mailing list archive

[Alex Meade <1433309@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

When many volumes are attached to instances on the same compute host
(with multipath enabled), volume detach is very slow and get slower as
more volumes are attached.

For example:
1. compute1 is a compute node with instance1 and instance2. 
2. instance1 has 10 volumes attached while instance2 has a single volume attached. 
3. Issue a detach for the volume attached to instance2
4. Nova spends >20 minutes executing the 'multipath -ll' command for every device on the hypervisor
5. Finally the detach completes successfully

The following log is output in n-cpu many, many times during the detach call. Repeated many times for each volume device.
http://paste.openstack.org/show/192981/


Environment details:
nova.conf virt driver
[libvirt]
iscsi_use_multipath = True
vif_driver = nova.virt.libvirt.vif.LibvirtGenericVIFDriver
inject_partition = -2
live_migration_uri = qemu+ssh://ameade@%s/system
use_usb_tablet = False
cpu_mode = none
virt_type = kvm

cinder.conf backend
[eseries]
volume_backend_name = eseries
volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver
netapp_storage_family = eseries
netapp_storage_protocol = iscsi
netapp_server_hostname = localhost
netapp_server_port = 8081
netapp_webservice_path = /devmgr/v2
netapp_controller_ips = 10.78.152.114,10.78.152.115
netapp_login = rw
netapp_password = xx
netapp_storage_pools = DDP
use_multipath_for_image_xfer = True
netapp_sa_password = password
netapp_enable_multi_attach=True

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1433309

Title:
  Libvirt: Detaching volume from instance on host with many attached
  volumes is very slow

Status in OpenStack Compute (Nova):
  New

Bug description:
  When many volumes are attached to instances on the same compute host
  (with multipath enabled), volume detach is very slow and get slower as
  more volumes are attached.

  For example:
  1. compute1 is a compute node with instance1 and instance2. 
  2. instance1 has 10 volumes attached while instance2 has a single volume attached. 
  3. Issue a detach for the volume attached to instance2
  4. Nova spends >20 minutes executing the 'multipath -ll' command for every device on the hypervisor
  5. Finally the detach completes successfully

  The following log is output in n-cpu many, many times during the detach call. Repeated many times for each volume device.
  http://paste.openstack.org/show/192981/

  
  Environment details:
  nova.conf virt driver
  [libvirt]
  iscsi_use_multipath = True
  vif_driver = nova.virt.libvirt.vif.LibvirtGenericVIFDriver
  inject_partition = -2
  live_migration_uri = qemu+ssh://ameade@%s/system
  use_usb_tablet = False
  cpu_mode = none
  virt_type = kvm

  cinder.conf backend
  [eseries]
  volume_backend_name = eseries
  volume_driver = cinder.volume.drivers.netapp.common.NetAppDriver
  netapp_storage_family = eseries
  netapp_storage_protocol = iscsi
  netapp_server_hostname = localhost
  netapp_server_port = 8081
  netapp_webservice_path = /devmgr/v2
  netapp_controller_ips = 10.78.152.114,10.78.152.115
  netapp_login = rw
  netapp_password = xx
  netapp_storage_pools = DDP
  use_multipath_for_image_xfer = True
  netapp_sa_password = password
  netapp_enable_multi_attach=True

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1433309/+subscriptions