yahoo-eng-team team mailing list archive

[John Garbutt <john@xxxxxxxxxxxxxxx>]

Looks like this is invalid now os-brick has merged.

** Tags added: volumes

** Changed in: nova
       Status: In Progress => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1452032

Title:
  Device descriptor not removed with different iqn and multipath
  enabled.

Status in OpenStack Compute (nova):
  Invalid
Status in Ubuntu:
  New

Bug description:
  [Environment]

  OpenStack Kilo
  Trusty 14.04.4

  [Description]

  if the attached multipath devices doesn't have same iqn like regular
  lvm+iscsi backend, in_use will be false.

  In that case,_disconnect_volume_multipath_iscsi() returns without
  calling _remove_multipath_device_descriptor().

  [Reproduction]

  1) Enable cinder LVM ISCSI on /etc/cinder/cinder.conf

  volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver

  2) Enable iscsi_use_multipath on /etc/nova/nova.conf on your compute
  nodes:

  iscsi_use_multipath = True

  3) Create 3 cinder volumes

  $ cinder create 1
  $ cinder create 1
  $ cinder create 1

  $ cinder list

  ubuntu@niedbalski2-bastion:~/specs/1374999$ cinder list
  +--------------------------------------+----------+--------------+------+-------------+----------+--------------------------------------+
  |                  ID                  |  Status  | Display Name | Size | Volume Type | Bootable |             Attached to              |
  +--------------------------------------+----------+--------------+------+-------------+----------+--------------------------------------+
  | 10844be6-8f86-414f-a10e-e1a31e2ba6e7 |  in-use  |     None     |  1   |     None    |  false   | b0a14447-5740-408a-b96f-a1e904b229e5 |
  | 1648d24c-0d65-4377-9fa5-6d3aeb8b1291 |  in-use  |     None     |  1   |     None    |  false   | b0a14447-5740-408a-b96f-a1e904b229e5 |
  | 53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f |  in-use  |     None     |  1   |     None    |  false   | b0a14447-5740-408a-b96f-a1e904b229e5 |
  +--------------------------------------+----------+--------------+------+-------------+----------+--------------------------------------+

  4) Attach them to nova

  $ nova volume-attach instance_id 10844be6-8f86-414f-a10e-e1a31e2ba6e7
  $ nova volume-attach instance_id 1648d24c-0d65-4377-9fa5-6d3aeb8b1291
  $ nova volume-attach instance_id 53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f

  5) Check on the nova-compute unit for the current multipath/session
  status

  tcp: [1] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-10844be6-8f86-414f-a10e-e1a31e2ba6e7
  tcp: [2] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-1648d24c-0d65-4377-9fa5-6d3aeb8b1291
  tcp: [3] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f

  Multipath:

  root@juju-1374999-machine-10:/home/ubuntu# multipath -ll
  33000000300000001 dm-2 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=1 status=active
    `- 10:0:0:1 sdb 8:16   active ready  running
  33000000100000001 dm-0 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=1 status=active
    `- 8:0:0:1  sdg 8:96   active ready  running
  33000000200000001 dm-1 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=1 status=active
    `- 9:0:0:1  sda 8:0    active ready  running

  6) Detach the current volumes.

  First.

  ubuntu@niedbalski2-bastion:~/specs/1374999$ nova volume-detach
  b0a14447-5740-408a-b96f-a1e904b229e5 10844be6-8f86-414f-a10e-
  e1a31e2ba6e7

  ubuntu@niedbalski2-bastion:~/specs/1374999$ juju ssh 10 "sudo multipath -ll"
  33000000300000001 dm-2 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=1 status=active
    `- 10:0:0:1 sdb 8:16   active ready  running
  33000000100000001 dm-0 ,
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=0 status=active
    `- #:#:#:#  -   #:#    active faulty running
  33000000200000001 dm-1 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=1 status=active
    `- 9:0:0:1  sda 8:0    active ready  running

  Second raises the faulty state

  33000000300000001 dm-2 IET,VIRTUAL-DISK
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=1 status=active
    `- 10:0:0:1 sdb 8:16   active ready  running
  33000000100000001 dm-0 ,
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=0 status=active
    `- #:#:#:#  -   #:#    active faulty running
  33000000200000001 dm-1 ,
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=0 status=active
    `- #:#:#:#  -   #:#    active faulty running

  Third, raises the faulty state also

  sudo: unable to resolve host juju-1374999-machine-10
  33000000300000001 dm-2 ,
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=0 status=active
    `- #:#:#:# -   #:#    active faulty running
  33000000100000001 dm-0 ,
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=0 status=active
    `- #:#:#:# -   #:#    active faulty running
  33000000200000001 dm-1 ,
  size=1.0G features='0' hwhandler='0' wp=rw
  `-+- policy='round-robin 0' prio=0 status=active
    `- #:#:#:# -   #:#    active faulty running

  
  Manual fix.

  root@juju-1374999-machine-10:/home/ubuntu# multipath -F 33000000300000001
  root@juju-1374999-machine-10:/home/ubuntu# multipath -F 33000000100000001
  root@juju-1374999-machine-10:/home/ubuntu# multipath -F 33000000200000001

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1452032/+subscriptions