yahoo-eng-team team mailing list archive

[Takashi Kajinami <1978444@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Description
===========
When cinder-api is running behind load balancer like haproxy, the load balancer can return 504 if it can not receive response from cinder-api within timeout.
When this timeout occurs while detaching a volume, this results in un-detachable volume.

 - nova-compute calls delete attachment api in cinder
 - haproxy detects server timeout and returns 504
 - cinder continues processing the API and removes the attachment
 - nova-compute immediately aborts the volume detachment and leaves the bdm
 - when a client tries to detach the volume again, the detachment fails because the attachment no longer exists in Nova

See for details https://bugzilla.redhat.com/show_bug.cgi?id=2002643

Steps to reproduce
==================
* Stop cinder-volume
* Detach a volume from an instance
* Start cinder-volume
* Detach the volume again

Expected result
===============
* Volume can be detached after cinder-volume is recovered

Actual result
===============
* Volume can't be detached

Environment
===========
* The issue was initially found in stable/train

Logs & Configs
==============
* See https://bugzilla.redhat.com/show_bug.cgi?id=2002643#c1

** Affects: nova
     Importance: Undecided
     Assignee: Takashi Kajinami (kajinamit)
         Status: New

** Changed in: nova
     Assignee: (unassigned) => Takashi Kajinami (kajinamit)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1978444

Title:
  Volume can't be detached if attachment delete api call fails with 504
  gateway timeout

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========
  When cinder-api is running behind load balancer like haproxy, the load balancer can return 504 if it can not receive response from cinder-api within timeout.
  When this timeout occurs while detaching a volume, this results in un-detachable volume.

   - nova-compute calls delete attachment api in cinder
   - haproxy detects server timeout and returns 504
   - cinder continues processing the API and removes the attachment
   - nova-compute immediately aborts the volume detachment and leaves the bdm
   - when a client tries to detach the volume again, the detachment fails because the attachment no longer exists in Nova

  See for details https://bugzilla.redhat.com/show_bug.cgi?id=2002643

  Steps to reproduce
  ==================
  * Stop cinder-volume
  * Detach a volume from an instance
  * Start cinder-volume
  * Detach the volume again

  Expected result
  ===============
  * Volume can be detached after cinder-volume is recovered

  Actual result
  ===============
  * Volume can't be detached

  Environment
  ===========
  * The issue was initially found in stable/train

  Logs & Configs
  ==============
  * See https://bugzilla.redhat.com/show_bug.cgi?id=2002643#c1

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1978444/+subscriptions