yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #72264
[Bug 1762687] Re: Concurrent requests to attach the same non-multiattach volume to multiple instances can succeed
** Changed in: nova
Status: New => Invalid
** No longer affects: nova
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1762687
Title:
Concurrent requests to attach the same non-multiattach volume to
multiple instances can succeed
Status in Cinder:
In Progress
Bug description:
Description
===========
Discovered this by chance yesterday, at first glance this appears to
be due to a lack of locking within c-api when we initial create the
attachments and no additional validation when we update with the
connector later in the attach flow. Reporting this against both nova
and cinder for now.
$ nova volume-attach 77c092c2-9664-42b2-bf71-277b1bfad707 b4240f39-da7a-4372-b4ca-15a0c6121ac8 & nova volume-attach 91a7c490-5a9a-4048-a109-b1159b7f0e79 b4240f39-da7a-4372-b4ca-15a0c6121ac8
[1] 24949
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | b4240f39-da7a-4372-b4ca-15a0c6121ac8 |
| serverId | 91a7c490-5a9a-4048-a109-b1159b7f0e79 |
| volumeId | b4240f39-da7a-4372-b4ca-15a0c6121ac8 |
+----------+--------------------------------------+
+----------+--------------------------------------+
| Property | Value |
+----------+--------------------------------------+
| device | /dev/vdb |
| id | b4240f39-da7a-4372-b4ca-15a0c6121ac8 |
| serverId | 77c092c2-9664-42b2-bf71-277b1bfad707 |
| volumeId | b4240f39-da7a-4372-b4ca-15a0c6121ac8 |
+----------+--------------------------------------+
$ cinder show b4240f39-da7a-4372-b4ca-15a0c6121ac8
+--------------------------------+----------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------+----------------------------------------------------------------------------------+
| attached_servers | ['91a7c490-5a9a-4048-a109-b1159b7f0e79', '77c092c2-9664-42b2-bf71-277b1bfad707'] |
| attachment_ids | ['31b8c16f-07d0-4f0c-95d8-c56797a270dc', 'a7eb9cb1-b7be-44e3-a176-3c6989459aaa'] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2018-04-09T19:02:46.000000 |
| description | None |
| encrypted | False |
| id | b4240f39-da7a-4372-b4ca-15a0c6121ac8 |
| metadata | attached_mode : rw |
| migration_status | None |
| multiattach | False |
| name | None |
| os-vol-host-attr:host | test.example.com@lvmdriver-1#lvmdriver-1 |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | fe3128ecf4704369ae3f7ede03f6bc29 |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| updated_at | 2018-04-09T19:04:24.000000 |
| user_id | 57293b0839da449580ce7008c8734c1c |
| volume_type | lvmdriver-1 |
+--------------------------------+----------------------------------------------------------------------------------+
$ ll /dev/disk/by-path/ip-192.168.122.79:3260-iscsi-iqn.2010-10.org.openstack:volume-b4240f39-da7a-4372-b4ca-15a0c6121ac8-lun-0
lrwxrwxrwx. 1 root root 9 Apr 9 15:04 /dev/disk/by-path/ip-192.168.122.79:3260-iscsi-iqn.2010-10.org.openstack:volume-b4240f39-da7a-4372-b4ca-15a0c6121ac8-lun-0 -> ../../sdc
$ sudo virsh domblklist 77c092c2-9664-42b2-bf71-277b1bfad707
Target Source
------------------------------------------------
vda /opt/stack/data/nova/instances/77c092c2-9664-42b2-bf71-277b1bfad707/disk
vdb /dev/sdc
$ sudo virsh domblklist 91a7c490-5a9a-4048-a109-b1159b7f0e79
Target Source
------------------------------------------------
vda /opt/stack/data/nova/instances/91a7c490-5a9a-4048-a109-b1159b7f0e79/disk
vdb /dev/sdc
Steps to reproduce
==================
$ nova volume-attach $instance_1_uuid $volume_uuid & nova volume-attach $instance_2_uuid $volume_uuid
Expected result
===============
Only one of the requests succeeds.
Actual result
=============
Both requests succeed.
Environment
===========
1. Exact version of OpenStack you are running. See the following
list for all releases: http://docs.openstack.org/releases/
$ cd ../nova
$ git describe --tags
17.0.0.0rc1-656-gebb4817ce3
$ cd ../cinder/
$ git describe --tags
12.0.0.0rc1-365-ga8a9dda30
2. Which hypervisor did you use?
(For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...)
What's the version of that?
Libvirt + KVM
2. Which storage type did you use?
(For example: Ceph, LVM, GPFS, ...)
What's the version of that?
LVM + iSCSI
3. Which networking type did you use?
(For example: nova-network, Neutron with OpenVSwitch, ...)
N/A
To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1762687/+subscriptions
References
