yahoo-eng-team team mailing list archive

[Mike Perez <thingee@xxxxxxxxx>]

** Changed in: cinder
       Status: In Progress => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1432963

Title:
  Volume of 'in-use' remain by a timeout during the attach

Status in Cinder:
  Invalid
Status in OpenStack Compute (nova):
  In Progress

Bug description:
  Remain as the 'in-use' volume due to timeout during the 'attach'.
  If the problem occurs, The instance can't detach/attach the volume.
  	detach -> volume_id not found
  	attach -> libvirtError: Requested operation is not valid: target vdb already exists 
  	(other volume)

  This problems are caused by the mismatching DB because c-vol does not rollback it when happend the timeout of RPC(CALL).
  At first, c-vol takes time in the attaching process, and the attaching process failed in the c-api by timeout.
  But in this time, the attaching process does not failed yet in the c-vol.

  By this, BDM is deleted and volume is updated to in-use.
  (If successful the attaching process in the c-vol.)

  <Repro> 
  used master
  	cinder: commit d4b77484c5d41f207d54f40dcdd530fb8a1b1ea6
  	nova  : commit eaeecdaf4743463888c3ee24fb08128eac15dee7

  1. attach volume
     (in cinder/volume/manage.py def attach_volume)
      note: I reproduced this problem by inserting sleep in this method.
  2. RPC(CALL) timeout in the c-api(attach_volume process takes time beyond the rpc_response_timeout)

  <About the volume and BDM >
  block_device_mapping
  +---------------------+----+-------------+-----------------------+--------------------------------------+-----------------+--------------------------------------+---------+
  | deleted_at          | id | device_name | delete_on_termination | volume_id                            | connection_info | instance_uuid                        | deleted |
  +---------------------+----+-------------+-----------------------+--------------------------------------+-----------------+--------------------------------------+---------+
  | NULL                |  1 | /dev/vda    |                     1 | NULL                                 | NULL            | 4683d4fb-758c-459e-9def-b8d247a56954 |       0 |
  | 2015-03-17 06:12:36 |  2 | /dev/vdb    |                     0 | 46d1bfbb-bdf2-472f-8bf6-2d2367b1edb1 | NULL            | 4683d4fb-758c-459e-9def-b8d247a56954 |       2 |
  +---------------------+----+-------------+-----------------------+--------------------------------------+-----------------+--------------------------------------+---------+
  volumes
  +------------+---------+--------------------------------------+--------+---------------+---------------+-----------------------------------------------------------------------------------------------+--------------------------------------------+
  | deleted_at | deleted | id                                   | status | attach_status | terminated_at | provider_location                                                                             | provider_auth                              |
  +------------+---------+--------------------------------------+--------+---------------+---------------+-----------------------------------------------------------------------------------------------+--------------------------------------------+
  | NULL       |       0 | 46d1bfbb-bdf2-472f-8bf6-2d2367b1edb1 | in-use | attached      | NULL          | 192.168.58.172:3260,4 iqn.2010-10.org.openstack:volume-46d1bfbb-bdf2-472f-8bf6-2d2367b1edb1 1 | CHAP 2W7r5XQcZJ5BHVctM8YY NogCxmq4VswXWHWE |
  +------------+---------+--------------------------------------+--------+---------------+---------------+-----------------------------------------------------------------------------------------------+--------------------------------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1432963/+subscriptions