yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #33151
[Bug 1459073] [NEW] Multipath device descriptor is not deleted after volume detached when FC and multipath are used
Public bug reported:
When Fibre Channel and multipath are used, the multipath device
descriptor is not deleted after a volume is detached from a VM. This is
always reproducible.
This was seen with git stable/kilo.
Reproduce steps:
1. Check the multipath device on the system
stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll
3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume
size=222G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:1:0 sdb 8:16 active ready running
2. Attach a volume to a VM runing on the current system, and wait till the volume becomes to 'in-use'
stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-attach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | fa7c94ae-e666-4923-be2b-8965036dce1a |
| serverId | c6244601-73e0-4210-8654-356a4154883b |
| volumeId | fa7c94ae-e666-4923-be2b-8965036dce1a |
+----------+--------------------------------------+
stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list
+--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+
| fa7c94ae-e666-4923-be2b-8965036dce1a | in-use | vol1 | 1 | None | false | c6244601-73e0-4210-8654-356a4154883b |
+--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+
3. Check the multipath devices on the system. 3600601602ba03400efd6d0247c03e511 is the multipath device mapping to the attached volume
stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll
3600601602ba03400efd6d0247c03e511 dm-1 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| `- 7:0:4:225 sdp 8:240 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 7:0:2:225 sdo 8:224 active ready running
3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume
size=222G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:1:0 sdb 8:16 active ready running
4. Detach the volume and wait till the volume becomes 'available'
stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-detach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a
stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
| fa7c94ae-e666-4923-be2b-8965036dce1a | available | vol1 | 1 | None | false | |
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
5. Check the multipath devices again. We can see the descriptor 3600601602ba03400efd6d0247c03e511 still exists
stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600601602ba03400efd6d0247c03e511
3600601602ba03400efd6d0247c03e511 dm-1 ,
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=-1 status=active
| `- #:#:#:# - #:# active undef running
`-+- policy='round-robin 0' prio=-1 status=enabled
`- #:#:#:# - #:# active undef running
================
Code Version:
stack@ubuntu-server12:/opt/stack/nova$ git status
On branch stable/kilo
Your branch is up-to-date with 'origin/stable/kilo'.
nothing to commit, working directory clean
stack@ubuntu-server12:/opt/stack/nova$ git log -1
commit 36fb00291d819b65f46d530eefbf07b883ca8a29
Merge: 0c60aca 8e9ecf7
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date: Fri May 15 16:16:00 2015 +0000
Merge "Libvirt: Use tpool to invoke guestfs api" into stable/kilo
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1459073
Title:
Multipath device descriptor is not deleted after volume detached when
FC and multipath are used
Status in OpenStack Compute (Nova):
New
Bug description:
When Fibre Channel and multipath are used, the multipath device
descriptor is not deleted after a volume is detached from a VM. This
is always reproducible.
This was seen with git stable/kilo.
Reproduce steps:
1. Check the multipath device on the system
stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll
3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume
size=222G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:1:0 sdb 8:16 active ready running
2. Attach a volume to a VM runing on the current system, and wait till the volume becomes to 'in-use'
stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-attach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | fa7c94ae-e666-4923-be2b-8965036dce1a |
| serverId | c6244601-73e0-4210-8654-356a4154883b |
| volumeId | fa7c94ae-e666-4923-be2b-8965036dce1a |
+----------+--------------------------------------+
stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list
+--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+
| fa7c94ae-e666-4923-be2b-8965036dce1a | in-use | vol1 | 1 | None | false | c6244601-73e0-4210-8654-356a4154883b |
+--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+
3. Check the multipath devices on the system. 3600601602ba03400efd6d0247c03e511 is the multipath device mapping to the attached volume
stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll
3600601602ba03400efd6d0247c03e511 dm-1 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=130 status=active
| `- 7:0:4:225 sdp 8:240 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 7:0:2:225 sdo 8:224 active ready running
3600508e000000000c57de73d39c00b0e dm-0 LSI,Logical Volume
size=222G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
`- 6:1:1:0 sdb 8:16 active ready running
4. Detach the volume and wait till the volume becomes 'available'
stack@ubuntu-server12:/proc/sys/dev/scsi$ nova volume-detach c6244601-73e0-4210-8654-356a4154883b fa7c94ae-e666-4923-be2b-8965036dce1a
stack@ubuntu-server12:/proc/sys/dev/scsi$ cinder list
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
| ID | Status | Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
| fa7c94ae-e666-4923-be2b-8965036dce1a | available | vol1 | 1 | None | false | |
+--------------------------------------+-----------+-----------+------+-------------+----------+-------------+
5. Check the multipath devices again. We can see the descriptor 3600601602ba03400efd6d0247c03e511 still exists
stack@ubuntu-server12:/proc/sys/dev/scsi$ sudo multipath -ll 3600601602ba03400efd6d0247c03e511
3600601602ba03400efd6d0247c03e511 dm-1 ,
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=-1 status=active
| `- #:#:#:# - #:# active undef running
`-+- policy='round-robin 0' prio=-1 status=enabled
`- #:#:#:# - #:# active undef running
================
Code Version:
stack@ubuntu-server12:/opt/stack/nova$ git status
On branch stable/kilo
Your branch is up-to-date with 'origin/stable/kilo'.
nothing to commit, working directory clean
stack@ubuntu-server12:/opt/stack/nova$ git log -1
commit 36fb00291d819b65f46d530eefbf07b883ca8a29
Merge: 0c60aca 8e9ecf7
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date: Fri May 15 16:16:00 2015 +0000
Merge "Libvirt: Use tpool to invoke guestfs api" into stable/kilo
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1459073/+subscriptions
Follow ups
References
