yahoo-eng-team team mailing list archive

[Rikimaru Honjo <honjo.rikimaru@xxxxxxxxxxxxx>]

Public bug reported:

I executed volume-attach API(nova V2 API) when RabbitMQ was down.
As result of above API execution, volume-attach API was failed and volume's status is still available.
But, block device mapping record remains on nova DB.
This condition is inconsistency.

And, remained block device mapping record maybe cause some problems.
(I'm researching now.)

I used openstack juno-3.

------------------------------------------------------------------------------------------------------------------
* Before executing volume-attach API:

$ nova list
+--------------------------------------+--------+--------+------------+-------------+--------------------+
| ID                                   | Name   | Status | Task State | Power State | Networks           |
+--------------------------------------+--------+--------+------------+-------------+--------------------+
| 0b529526-4c8d-4650-8295-b7155a977ba7 | testVM | ACTIVE | -          | Running     | private=10.0.0.104 |
+--------------------------------------+--------+--------+------------+-------------+--------------------+
$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| e93478bf-ee37-430f-93df-b3cf26540212 | available |     None     |  1   |     None    |  false   |             |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
devstack@ubuntu-14-04-01-64-juno3-01:~$

mysql> select * from block_device_mapping where instance_uuid = '0b529526-4c8d-4650-8295-b7155a977ba7';
+---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+-----------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
| created_at          | updated_at          | deleted_at | id  | device_name | delete_on_termination | snapshot_id | volume_id | volume_size | no_device | connection_info | instance_uuid                        | deleted | source_type | destination_type | guest_format | device_type | disk_bus | boot_index | image_id                             |
+---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+-----------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
| 2014-10-02 18:36:08 | 2014-10-02 18:36:10 | NULL       | 145 | /dev/vda    |                     1 | NULL        | NULL      |        NULL |      NULL | NULL            | 0b529526-4c8d-4650-8295-b7155a977ba7 |       0 | image       | local            | NULL         | disk        | NULL     |          0 | c1d264fd-c559-446e-9b94-934ba8249ae1 |
+---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+-----------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
1 row in set (0.00 sec)

* After executing volume-attach API:
$ nova list --all-t
+--------------------------------------+--------+--------+------------+-------------+--------------------+
| ID                                   | Name   | Status | Task State | Power State | Networks           |
+--------------------------------------+--------+--------+------------+-------------+--------------------+
| 0b529526-4c8d-4650-8295-b7155a977ba7 | testVM | ACTIVE | -          | Running     | private=10.0.0.104 |
+--------------------------------------+--------+--------+------------+-------------+--------------------+
$ cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| e93478bf-ee37-430f-93df-b3cf26540212 | available |     None     |  1   |     None    |  false   |             |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

mysql> select * from block_device_mapping where instance_uuid = '0b529526-4c8d-4650-8295-b7155a977ba7';
+---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
| created_at          | updated_at          | deleted_at | id  | device_name | delete_on_termination | snapshot_id | volume_id                            | volume_size | no_device | connection_info | instance_uuid                        | deleted | source_type | destination_type | guest_format | device_type | disk_bus | boot_index | image_id                             |
+---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
| 2014-10-02 18:36:08 | 2014-10-02 18:36:10 | NULL       | 145 | /dev/vda    |                     1 | NULL        | NULL                                 |        NULL |      NULL | NULL            | 0b529526-4c8d-4650-8295-b7155a977ba7 |       0 | image       | local            | NULL         | disk        | NULL     |          0 | c1d264fd-c559-446e-9b94-934ba8249ae1 |
| 2014-10-02 18:45:28 | NULL                | NULL       | 146 | /dev/vdb    |                     0 | NULL        | 0b529526-4c8d-4650-8295-b7155a977ba7 |        NULL |      NULL | NULL            | 0b529526-4c8d-4650-8295-b7155a977ba7 |       0 | volume      | volume           | NULL         | NULL        | NULL     |       NULL | NULL                                 |
+---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
2 rows in set (0.00 sec)

------------------------------------------------------------------------------------------------------------------

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1377161

Title:
  If volume-attach API is failed, Block Device Mapping record will
  remain

Status in OpenStack Compute (Nova):
  New

Bug description:
  I executed volume-attach API(nova V2 API) when RabbitMQ was down.
  As result of above API execution, volume-attach API was failed and volume's status is still available.
  But, block device mapping record remains on nova DB.
  This condition is inconsistency.

  And, remained block device mapping record maybe cause some problems.
  (I'm researching now.)

  I used openstack juno-3.

  ------------------------------------------------------------------------------------------------------------------
  * Before executing volume-attach API:

  $ nova list
  +--------------------------------------+--------+--------+------------+-------------+--------------------+
  | ID                                   | Name   | Status | Task State | Power State | Networks           |
  +--------------------------------------+--------+--------+------------+-------------+--------------------+
  | 0b529526-4c8d-4650-8295-b7155a977ba7 | testVM | ACTIVE | -          | Running     | private=10.0.0.104 |
  +--------------------------------------+--------+--------+------------+-------------+--------------------+
  $ cinder list
  +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
  |                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
  | e93478bf-ee37-430f-93df-b3cf26540212 | available |     None     |  1   |     None    |  false   |             |
  +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
  devstack@ubuntu-14-04-01-64-juno3-01:~$

  mysql> select * from block_device_mapping where instance_uuid = '0b529526-4c8d-4650-8295-b7155a977ba7';
  +---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+-----------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
  | created_at          | updated_at          | deleted_at | id  | device_name | delete_on_termination | snapshot_id | volume_id | volume_size | no_device | connection_info | instance_uuid                        | deleted | source_type | destination_type | guest_format | device_type | disk_bus | boot_index | image_id                             |
  +---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+-----------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
  | 2014-10-02 18:36:08 | 2014-10-02 18:36:10 | NULL       | 145 | /dev/vda    |                     1 | NULL        | NULL      |        NULL |      NULL | NULL            | 0b529526-4c8d-4650-8295-b7155a977ba7 |       0 | image       | local            | NULL         | disk        | NULL     |          0 | c1d264fd-c559-446e-9b94-934ba8249ae1 |
  +---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+-----------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
  1 row in set (0.00 sec)

  * After executing volume-attach API:
  $ nova list --all-t
  +--------------------------------------+--------+--------+------------+-------------+--------------------+
  | ID                                   | Name   | Status | Task State | Power State | Networks           |
  +--------------------------------------+--------+--------+------------+-------------+--------------------+
  | 0b529526-4c8d-4650-8295-b7155a977ba7 | testVM | ACTIVE | -          | Running     | private=10.0.0.104 |
  +--------------------------------------+--------+--------+------------+-------------+--------------------+
  $ cinder list
  +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
  |                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
  +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
  | e93478bf-ee37-430f-93df-b3cf26540212 | available |     None     |  1   |     None    |  false   |             |
  +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

  mysql> select * from block_device_mapping where instance_uuid = '0b529526-4c8d-4650-8295-b7155a977ba7';
  +---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
  | created_at          | updated_at          | deleted_at | id  | device_name | delete_on_termination | snapshot_id | volume_id                            | volume_size | no_device | connection_info | instance_uuid                        | deleted | source_type | destination_type | guest_format | device_type | disk_bus | boot_index | image_id                             |
  +---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
  | 2014-10-02 18:36:08 | 2014-10-02 18:36:10 | NULL       | 145 | /dev/vda    |                     1 | NULL        | NULL                                 |        NULL |      NULL | NULL            | 0b529526-4c8d-4650-8295-b7155a977ba7 |       0 | image       | local            | NULL         | disk        | NULL     |          0 | c1d264fd-c559-446e-9b94-934ba8249ae1 |
  | 2014-10-02 18:45:28 | NULL                | NULL       | 146 | /dev/vdb    |                     0 | NULL        | 0b529526-4c8d-4650-8295-b7155a977ba7 |        NULL |      NULL | NULL            | 0b529526-4c8d-4650-8295-b7155a977ba7 |       0 | volume      | volume           | NULL         | NULL        | NULL     |       NULL | NULL                                 |
  +---------------------+---------------------+------------+-----+-------------+-----------------------+-------------+--------------------------------------+-------------+-----------+-----------------+--------------------------------------+---------+-------------+------------------+--------------+-------------+----------+------------+--------------------------------------+
  2 rows in set (0.00 sec)

  ------------------------------------------------------------------------------------------------------------------

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1377161/+subscriptions