yahoo-eng-team team mailing list archive

[Sean Severson <sseverson@xxxxxx>]

Public bug reported:

When running long tests involving the continuous attach and detach of
hundreds of consecutive volumes, Nova will blindly assign them
incrementing device paths and never reuse paths that were freed up. This
eventually leads to strings of errors in n-api such as the following:

2014-10-02 14:04:12.298 DEBUG nova.api.openstack.wsgi [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] Action: 'create', calling method: <bound method VolumeAttachmentController.create of <nova.api.openstack.compute.contrib.volumes.VolumeAttachmentController object at 0x7fe62b01a150>>, body: {"volumeAttachment": {"device": "/dev/vd~w", "volumeId": "ccbc09e9-50a4-46b9-a413-66db51659abe"}} from (pid=48499) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:908
2014-10-02 14:04:12.299 AUDIT nova.api.openstack.compute.contrib.volumes [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] Attach volume ccbc09e9-50a4-46b9-a413-66db51659abe to instance dfbfe4a2-19fb-4f01-be1a-9e437aec67df at /dev/vd~w
2014-10-02 14:04:12.358 DEBUG nova.api.openstack.wsgi [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] Returning 400 to user: The supplied device path (/dev/vd~w) is invalid. from (pid=48499) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1175
2014-10-02 14:04:12.359 INFO nova.osapi_compute.wsgi.server [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] 10.50.135.12 "POST /v2/4cb643dec0bd40c89b984dacfd288448/servers/dfbfe4a2-19fb-4f01-be1a-9e437aec67df/os-volume_attachments HTTP/1.1" status: 400 len: 277 time: 0.0690210

On the instance, the device paths mentioned in n-api aren't even being
used. Typically the instance will reuse /dev/vdb, assuming only one
volume at a time is being attached. Nova should be reporting the device
path that's actually being used, and should definitely not be using
special characters in the path.

Note that this is not a case of rapidly reattaching the same volume. To
properly reproduce this situation, it is necessary to attach and detach
new volumes.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1376942

Title:
  Rapid attach/detach of consecutive volumes eats up device names

Status in OpenStack Compute (Nova):
  New

Bug description:
  When running long tests involving the continuous attach and detach of
  hundreds of consecutive volumes, Nova will blindly assign them
  incrementing device paths and never reuse paths that were freed up.
  This eventually leads to strings of errors in n-api such as the
  following:

  2014-10-02 14:04:12.298 DEBUG nova.api.openstack.wsgi [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] Action: 'create', calling method: <bound method VolumeAttachmentController.create of <nova.api.openstack.compute.contrib.volumes.VolumeAttachmentController object at 0x7fe62b01a150>>, body: {"volumeAttachment": {"device": "/dev/vd~w", "volumeId": "ccbc09e9-50a4-46b9-a413-66db51659abe"}} from (pid=48499) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:908
  2014-10-02 14:04:12.299 AUDIT nova.api.openstack.compute.contrib.volumes [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] Attach volume ccbc09e9-50a4-46b9-a413-66db51659abe to instance dfbfe4a2-19fb-4f01-be1a-9e437aec67df at /dev/vd~w
  2014-10-02 14:04:12.358 DEBUG nova.api.openstack.wsgi [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] Returning 400 to user: The supplied device path (/dev/vd~w) is invalid. from (pid=48499) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1175
  2014-10-02 14:04:12.359 INFO nova.osapi_compute.wsgi.server [req-16e36730-3f4c-4f75-8107-10d4ecfba293 admin admin] 10.50.135.12 "POST /v2/4cb643dec0bd40c89b984dacfd288448/servers/dfbfe4a2-19fb-4f01-be1a-9e437aec67df/os-volume_attachments HTTP/1.1" status: 400 len: 277 time: 0.0690210

  On the instance, the device paths mentioned in n-api aren't even being
  used. Typically the instance will reuse /dev/vdb, assuming only one
  volume at a time is being attached. Nova should be reporting the
  device path that's actually being used, and should definitely not be
  using special characters in the path.

  Note that this is not a case of rapidly reattaching the same volume.
  To properly reproduce this situation, it is necessary to attach and
  detach new volumes.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1376942/+subscriptions