← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1522705] Re: Cinder volumes are stuck when non admin user executes nova swap volume API

 

Reviewed:  https://review.openstack.org/299715
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=f738483e843fc27379b85c5401859ccc854adc5e
Submitter: Jenkins
Branch:    master

commit f738483e843fc27379b85c5401859ccc854adc5e
Author: Matt Riedemann <mriedem@xxxxxxxxxx>
Date:   Wed Mar 30 22:07:47 2016 -0400

    Make swap-volume an admin-only API by default
    
    Cinder's volume migration API is, by default, an admin-only operation.
    This includes the migrate_volume_completion API.
    
    When Cinder is doing a volume migration, it calls Nova's swap-volume
    API to detach the old volume that we're migrating from and attach
    the volume that we're migrating to. Then Nova calls Cinder's
    migrate_volume_completion API to signal Nova is done and Cinder
    can finish the volume migration.
    
    The problem is that swap-volume is not an admin-only API in Nova
    per the default policy. So if a non-admin user tries to perform
    a swap-volume operation, it will fail with a 403 when calling
    Cinder's migrate_volume_completion API, since that requires an
    admin user.
    
    Also, because of 98739761f17b5e0b32abd8cd262f5beda030f886 we can't
    simply avoid calling migrate_volume_completion for non-migration
    cases because that API handles the actual detach/attach for the old
    and new volumes, swap-volume is broken without calling that.
    
    So given swap-volume relies on an admin-only Cinder API, and is called
    from an admin-only Cinder operation (volume migration), we should
    just make it default to admin-only also.
    
    Change-Id: Iac03258735f3d856a474ab96fe9b0a087e32906f
    Closes-Bug: #1522705


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1522705

Title:
  Cinder volumes are stuck when non admin user executes nova swap volume
  API

Status in Cinder:
  Invalid
Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Cinder volumes are stuck in attaching/detaching state when non admin user executes nova swap volume API.
  Because cinder 'migrate_volume_completion' API can be executed by admin only in default settings of cinder policy.json.
  So the default settings of cinder policy.json should be fixed.

  [How to reproduce]
  stack@devstack-master:/opt/devstack$ env | grep OS
  (snipped...)
  OS_USERNAME=demo
  OS_TENANT_NAME=demo
  (snipped...)
  stack@devstack-master:/opt/devstack$ nova list
  +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+
  | ID                                   | Name    | Status | Task State | Power State | Networks                                                |
  +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+
  | 5a4102cd-9e36-480c-a148-d2a127ff704e | server1 | ACTIVE | -          | Running     | private=10.0.10.3, fd61:9f1e:73ec:0:f816:3eff:fe34:8b61 |
  +--------------------------------------+---------+--------+------------+-------------+---------------------------------------------------------+
  stack@devstack-master:/opt/devstack$ cinder list
  +--------------------------------------+-----------+-------+------+-------------+----------+-------------+--------------------------------------+
  |                  ID                  |   Status  |  Name | Size | Volume Type | Bootable | Multiattach |             Attached to              |
  +--------------------------------------+-----------+-------+------+-------------+----------+-------------+--------------------------------------+
  | 1c768de0-949f-485f-b914-c430752e2984 | available | TEST2 |  1   | lvmdriver-1 |  false   |    False    |                                      |
  | 999a256e-aadf-41b6-88f3-1412e3462cbe |   in-use  | TEST1 |  1   | lvmdriver-1 |  false   |    False    | 5a4102cd-9e36-480c-a148-d2a127ff704e |
  +--------------------------------------+-----------+-------+------+-------------+----------+-------------+--------------------------------------+
  stack@devstack-master:/opt/devstack$ nova volume-update server1 999a256e-aadf-41b6-88f3-1412e3462cbe 1c768de0-949f-485f-b914-c430752e2984
  stack@devstack-master:/opt/devstack$ cinder list
  +--------------------------------------+-----------+-------+------+-------------+----------+-------------+--------------------------------------+
  |                  ID                  |   Status  |  Name | Size | Volume Type | Bootable | Multiattach |             Attached to              |
  +--------------------------------------+-----------+-------+------+-------------+----------+-------------+--------------------------------------+
  | 1c768de0-949f-485f-b914-c430752e2984 | attaching | TEST2 |  1   | lvmdriver-1 |  false   |    False    |                                      |
  | 999a256e-aadf-41b6-88f3-1412e3462cbe | detaching | TEST1 |  1   | lvmdriver-1 |  false   |    False    | 5a4102cd-9e36-480c-a148-d2a127ff704e |
  +--------------------------------------+-----------+-------+------+-------------+----------+-------------+--------------------------------------+

  [cinder-api.log]
  2015-12-04 15:41:51.986 INFO cinder.api.openstack.wsgi [req-44d81e36-299f-4e14-9419-996a2239b1ca 181cd8bc1c004030aec893fc7de79618 722380ff5288483191cd2712cbc99c5d] POST http://10.0.2.15:8776/v2/722380ff5288483191cd2712cbc99c5d/volumes/999a256e-aadf-41b6-88f3-1412e3462cbe/action
  2015-12-04 15:41:51.986 DEBUG cinder.api.openstack.wsgi [req-44d81e36-299f-4e14-9419-996a2239b1ca 181cd8bc1c004030aec893fc7de79618 722380ff5288483191cd2712cbc99c5d] Action body: {"os-migrate_volume_completion": {"new_volume": "1c768de0-949f-485f-b914-c430752e2984", "error": false}} from (pid=18203) get_method /opt/stack/cinder/cinder/api/openstack/wsgi.py:1093
  2015-12-04 15:41:51.987 INFO cinder.api.openstack.wsgi [req-44d81e36-299f-4e14-9419-996a2239b1ca 181cd8bc1c004030aec893fc7de79618 722380ff5288483191cd2712cbc99c5d] http://10.0.2.15:8776/v2/722380ff5288483191cd2712cbc99c5d/volumes/999a256e-aadf-41b6-88f3-1412e3462cbe/action returned with HTTP 403
  2015-12-04 15:41:51.989 INFO eventlet.wsgi.server [req-44d81e36-299f-4e14-9419-996a2239b1ca 181cd8bc1c004030aec893fc7de79618 722380ff5288483191cd2712cbc99c5d] 10.0.2.15 - - [04/Dec/2015 15:41:51] "POST /v2/722380ff5288483191cd2712cbc99c5d/volumes/999a256e-aadf-41b6-88f3-1412e3462cbe/action HTTP/1.1" 403 429 0.005410

  [policy.json(cinder)]
  The follwong line defines that 'migrate_volume_completion' can be executed by admin only.

  "volume_extension:volume_admin_actions:migrate_volume_completion":
  "rule:admin_api",

  [Environment]
  nova: commit 7df427fdb4d7a314b79b1e3977aeb86f019b5186(master)
  cinder: commit 707902ca482d1653072c2cf419c566cd5fee1f96(master)
  OS: Ubuntu 14.04

  stack@devstack-master:/opt/devstack$ keystone user-role-list --user demo --tenant demo
  +----------------------------------+-------------+----------------------------------+----------------------------------+
  |                id                |     name    |             user_id              |            tenant_id             |
  +----------------------------------+-------------+----------------------------------+----------------------------------+
  | dec5e6b197d6424bbfb3854f6718cef0 |    Member   | 181cd8bc1c004030aec893fc7de79618 | 722380ff5288483191cd2712cbc99c5d |
  | de75e50dec2e48b4a4c66feef8f53432 | anotherrole | 181cd8bc1c004030aec893fc7de79618 | 722380ff5288483191cd2712cbc99c5d |
  +----------------------------------+-------------+----------------------------------+----------------------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/cinder/+bug/1522705/+subscriptions


References