← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1471098] [NEW] Cinder volume stuck in swap_volume

 

Public bug reported:

Cinder volumes are stuck in 'attaching'/'detaching' state when 'swap
volume' is executed.

A cinder volume is attached to a VM instance.
Then the cinder volume is swapped for a new volume by 'swap volume'.
Concretely the following API is called.

PUT /v2/{tenant_id}/servers/{server_id}/os-
volume_attachments/{attachment_id}

After the API is called, the attached volume(old volume) becomes stuck in 'detaching' state
and the new volume becomes stuck in 'attaching' state.

[How to reproduce]
stack@devstack-kilo:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
|                  ID                  |   Status  |  Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
| c3aff356-7545-444c-8b4f-33670b3e483c |   in-use  | TEST2 |  1   | lvmdriver-1 |  false   | adc00700-c7c8-4fe9-b3b4-df9beed40405 |
| da251e5b-a783-4f96-8b9f-a8db5dc070c1 | available | TEST1 |  1   | lvmdriver-1 |  false   |                                      |
+--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
stack@devstack-kilo:/opt/devstack$ nova show server1
+--------------------------------------+----------------------------------------------------------------+
| Property                             | Value                                                          |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                         |
| OS-EXT-AZ:availability_zone          | nova                                                           |
| OS-EXT-SRV-ATTR:host                 | devstack-kilo                                                  |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | devstack-kilo                                                  |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                              |
| OS-EXT-STS:power_state               | 1                                                              |
| OS-EXT-STS:task_state                | -                                                              |
| OS-EXT-STS:vm_state                  | active                                                         |
| OS-SRV-USG:launched_at               | 2015-07-02T00:53:25.000000                                     |
| OS-SRV-USG:terminated_at             | -                                                              |
| accessIPv4                           |                                                                |
| accessIPv6                           |                                                                |
| config_drive                         | True                                                           |
| created                              | 2015-07-02T00:53:19Z                                           |
| flavor                               | m1.tiny (1)                                                    |
| hostId                               | ea09a5e13b086e757a5c21f093c46d0aa6ae373d82a34ef7ac798816       |
| id                                   | adc00700-c7c8-4fe9-b3b4-df9beed40405                           |
| image                                | cirros-0.3.2-x86_64-uec (abe0afbf-7f82-4361-a308-a69d7206989f) |
| key_name                             | -                                                              |
| metadata                             | {}                                                             |
| name                                 | server1                                                        |
| os-extended-volumes:volumes_attached | [{"id": "c3aff356-7545-444c-8b4f-33670b3e483c"}]               |
| progress                             | 0                                                              |
| public network                       | 10.0.2.195                                                     |
| security_groups                      | default                                                        |
| status                               | ACTIVE                                                         |
| tenant_id                            | 252be0c183fa42f788947cd18004ae63                               |
| updated                              | 2015-07-02T00:53:25Z                                           |
| user_id                              | 4f870a0e7b4a420b874fcb3f9718f040                               |
+--------------------------------------+----------------------------------------------------------------+
stack@devstack-kilo:/opt/devstack$ curl -i -X PUT http://10.0.2.15:8774/v2/252be0c183fa42f788947cd18004ae63/servers/adc00700-c7c8-4fe9-b3b4-df9beed40405/os-volume_attachments/c3aff356-7545-444c-8b4f-33670b3e483c -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: e6f83c8ca44644a39d787fd7308f4920" -d '{"volumeAttachment": {"volumeId": "da251e5b-a783-4f96-8b9f-a8db5dc070c1"}}'
HTTP/1.1 202 Accepted
Content-Type: text/html; charset=UTF-8
Content-Length: 0
X-Compute-Request-Id: req-9300c544-95c1-4993-abb6-d0cd1a7bddcf
Date: Thu, 02 Jul 2015 00:59:16 GMT

stack@devstack-kilo:/opt/devstack$ cinder list
+--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
|                  ID                  |   Status  |  Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
| c3aff356-7545-444c-8b4f-33670b3e483c | detaching | TEST2 |  1   | lvmdriver-1 |  false   | adc00700-c7c8-4fe9-b3b4-df9beed40405 |
| da251e5b-a783-4f96-8b9f-a8db5dc070c1 | attaching | TEST1 |  1   | lvmdriver-1 |  false   |                                      |
+--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
stack@devstack-kilo:/opt/devstack$ nova show server1
+--------------------------------------+----------------------------------------------------------------+
| Property                             | Value                                                          |
+--------------------------------------+----------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                         |
| OS-EXT-AZ:availability_zone          | nova                                                           |
| OS-EXT-SRV-ATTR:host                 | devstack-kilo                                                  |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | devstack-kilo                                                  |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                              |
| OS-EXT-STS:power_state               | 1                                                              |
| OS-EXT-STS:task_state                | -                                                              |
| OS-EXT-STS:vm_state                  | active                                                         |
| OS-SRV-USG:launched_at               | 2015-07-02T00:53:25.000000                                     |
| OS-SRV-USG:terminated_at             | -                                                              |
| accessIPv4                           |                                                                |
| accessIPv6                           |                                                                |
| config_drive                         | True                                                           |
| created                              | 2015-07-02T00:53:19Z                                           |
| flavor                               | m1.tiny (1)                                                    |
| hostId                               | ea09a5e13b086e757a5c21f093c46d0aa6ae373d82a34ef7ac798816       |
| id                                   | adc00700-c7c8-4fe9-b3b4-df9beed40405                           |
| image                                | cirros-0.3.2-x86_64-uec (abe0afbf-7f82-4361-a308-a69d7206989f) |
| key_name                             | -                                                              |
| metadata                             | {}                                                             |
| name                                 | server1                                                        |
| os-extended-volumes:volumes_attached | [{"id": "da251e5b-a783-4f96-8b9f-a8db5dc070c1"}]               |
| progress                             | 0                                                              |
| public network                       | 10.0.2.195                                                     |
| security_groups                      | default                                                        |
| status                               | ACTIVE                                                         |
| tenant_id                            | 252be0c183fa42f788947cd18004ae63                               |
| updated                              | 2015-07-02T00:59:31Z                                           |
| user_id                              | 4f870a0e7b4a420b874fcb3f9718f040                               |
+--------------------------------------+----------------------------------------------------------------+

[Environment]
OS: Ubuntu 14.04 LTS
nova: stable/kilo(commit d22167e08388f6f557484e469ab559255cc273c2)
cinder: stable/kilo(commit 6c055943e70675585ef45301318c47ccc1139ce9)

** Affects: nova
     Importance: Undecided
     Assignee: Takashi NATSUME (natsume-takashi)
         Status: New

** Changed in: nova
     Assignee: (unassigned) => Takashi NATSUME (natsume-takashi)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1471098

Title:
  Cinder volume stuck in swap_volume

Status in OpenStack Compute (Nova):
  New

Bug description:
  Cinder volumes are stuck in 'attaching'/'detaching' state when 'swap
  volume' is executed.

  A cinder volume is attached to a VM instance.
  Then the cinder volume is swapped for a new volume by 'swap volume'.
  Concretely the following API is called.

  PUT /v2/{tenant_id}/servers/{server_id}/os-
  volume_attachments/{attachment_id}

  After the API is called, the attached volume(old volume) becomes stuck in 'detaching' state
  and the new volume becomes stuck in 'attaching' state.

  [How to reproduce]
  stack@devstack-kilo:/opt/devstack$ cinder list
  +--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
  |                  ID                  |   Status  |  Name | Size | Volume Type | Bootable |             Attached to              |
  +--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
  | c3aff356-7545-444c-8b4f-33670b3e483c |   in-use  | TEST2 |  1   | lvmdriver-1 |  false   | adc00700-c7c8-4fe9-b3b4-df9beed40405 |
  | da251e5b-a783-4f96-8b9f-a8db5dc070c1 | available | TEST1 |  1   | lvmdriver-1 |  false   |                                      |
  +--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
  stack@devstack-kilo:/opt/devstack$ nova show server1
  +--------------------------------------+----------------------------------------------------------------+
  | Property                             | Value                                                          |
  +--------------------------------------+----------------------------------------------------------------+
  | OS-DCF:diskConfig                    | MANUAL                                                         |
  | OS-EXT-AZ:availability_zone          | nova                                                           |
  | OS-EXT-SRV-ATTR:host                 | devstack-kilo                                                  |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | devstack-kilo                                                  |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                              |
  | OS-EXT-STS:power_state               | 1                                                              |
  | OS-EXT-STS:task_state                | -                                                              |
  | OS-EXT-STS:vm_state                  | active                                                         |
  | OS-SRV-USG:launched_at               | 2015-07-02T00:53:25.000000                                     |
  | OS-SRV-USG:terminated_at             | -                                                              |
  | accessIPv4                           |                                                                |
  | accessIPv6                           |                                                                |
  | config_drive                         | True                                                           |
  | created                              | 2015-07-02T00:53:19Z                                           |
  | flavor                               | m1.tiny (1)                                                    |
  | hostId                               | ea09a5e13b086e757a5c21f093c46d0aa6ae373d82a34ef7ac798816       |
  | id                                   | adc00700-c7c8-4fe9-b3b4-df9beed40405                           |
  | image                                | cirros-0.3.2-x86_64-uec (abe0afbf-7f82-4361-a308-a69d7206989f) |
  | key_name                             | -                                                              |
  | metadata                             | {}                                                             |
  | name                                 | server1                                                        |
  | os-extended-volumes:volumes_attached | [{"id": "c3aff356-7545-444c-8b4f-33670b3e483c"}]               |
  | progress                             | 0                                                              |
  | public network                       | 10.0.2.195                                                     |
  | security_groups                      | default                                                        |
  | status                               | ACTIVE                                                         |
  | tenant_id                            | 252be0c183fa42f788947cd18004ae63                               |
  | updated                              | 2015-07-02T00:53:25Z                                           |
  | user_id                              | 4f870a0e7b4a420b874fcb3f9718f040                               |
  +--------------------------------------+----------------------------------------------------------------+
  stack@devstack-kilo:/opt/devstack$ curl -i -X PUT http://10.0.2.15:8774/v2/252be0c183fa42f788947cd18004ae63/servers/adc00700-c7c8-4fe9-b3b4-df9beed40405/os-volume_attachments/c3aff356-7545-444c-8b4f-33670b3e483c -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: e6f83c8ca44644a39d787fd7308f4920" -d '{"volumeAttachment": {"volumeId": "da251e5b-a783-4f96-8b9f-a8db5dc070c1"}}'
  HTTP/1.1 202 Accepted
  Content-Type: text/html; charset=UTF-8
  Content-Length: 0
  X-Compute-Request-Id: req-9300c544-95c1-4993-abb6-d0cd1a7bddcf
  Date: Thu, 02 Jul 2015 00:59:16 GMT

  stack@devstack-kilo:/opt/devstack$ cinder list
  +--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
  |                  ID                  |   Status  |  Name | Size | Volume Type | Bootable |             Attached to              |
  +--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
  | c3aff356-7545-444c-8b4f-33670b3e483c | detaching | TEST2 |  1   | lvmdriver-1 |  false   | adc00700-c7c8-4fe9-b3b4-df9beed40405 |
  | da251e5b-a783-4f96-8b9f-a8db5dc070c1 | attaching | TEST1 |  1   | lvmdriver-1 |  false   |                                      |
  +--------------------------------------+-----------+-------+------+-------------+----------+--------------------------------------+
  stack@devstack-kilo:/opt/devstack$ nova show server1
  +--------------------------------------+----------------------------------------------------------------+
  | Property                             | Value                                                          |
  +--------------------------------------+----------------------------------------------------------------+
  | OS-DCF:diskConfig                    | MANUAL                                                         |
  | OS-EXT-AZ:availability_zone          | nova                                                           |
  | OS-EXT-SRV-ATTR:host                 | devstack-kilo                                                  |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | devstack-kilo                                                  |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000001                                              |
  | OS-EXT-STS:power_state               | 1                                                              |
  | OS-EXT-STS:task_state                | -                                                              |
  | OS-EXT-STS:vm_state                  | active                                                         |
  | OS-SRV-USG:launched_at               | 2015-07-02T00:53:25.000000                                     |
  | OS-SRV-USG:terminated_at             | -                                                              |
  | accessIPv4                           |                                                                |
  | accessIPv6                           |                                                                |
  | config_drive                         | True                                                           |
  | created                              | 2015-07-02T00:53:19Z                                           |
  | flavor                               | m1.tiny (1)                                                    |
  | hostId                               | ea09a5e13b086e757a5c21f093c46d0aa6ae373d82a34ef7ac798816       |
  | id                                   | adc00700-c7c8-4fe9-b3b4-df9beed40405                           |
  | image                                | cirros-0.3.2-x86_64-uec (abe0afbf-7f82-4361-a308-a69d7206989f) |
  | key_name                             | -                                                              |
  | metadata                             | {}                                                             |
  | name                                 | server1                                                        |
  | os-extended-volumes:volumes_attached | [{"id": "da251e5b-a783-4f96-8b9f-a8db5dc070c1"}]               |
  | progress                             | 0                                                              |
  | public network                       | 10.0.2.195                                                     |
  | security_groups                      | default                                                        |
  | status                               | ACTIVE                                                         |
  | tenant_id                            | 252be0c183fa42f788947cd18004ae63                               |
  | updated                              | 2015-07-02T00:59:31Z                                           |
  | user_id                              | 4f870a0e7b4a420b874fcb3f9718f040                               |
  +--------------------------------------+----------------------------------------------------------------+

  [Environment]
  OS: Ubuntu 14.04 LTS
  nova: stable/kilo(commit d22167e08388f6f557484e469ab559255cc273c2)
  cinder: stable/kilo(commit 6c055943e70675585ef45301318c47ccc1139ce9)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1471098/+subscriptions


Follow ups

References