← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #35008

[Bug 1471289] [NEW] Fernet tokens and Federated Identities result in token scope failures

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When keystone is configured to use fernet tokens and also configured to
be a SP for an external IDP then the token data received by nova and
other services appear to not contain the right information, resulting in
errors from nova-api-os-compute such as:

Returning 400 to user: Malformed request URL: URL's project_id
'69f5cff441e04554b285d7772630dec1' doesn't match Context's project_id
'None'

When keystone is switched to use uuid tokens, then everything works as
expected.

Further debugging of the request to the nova api shows:

'HTTP_X_USER_DOMAIN_NAME': None,
'HTTP_X_DOMAIN_ID': None,
'HTTP_X_PROJECT_DOMAIN_ID': None,
'HTTP_X_ROLES': '',
'HTTP_X_TENANT_ID': None,
'HTTP_X_PROJECT_DOMAIN_NAME': None,
'HTTP_X_TENANT': None,
'HTTP_X_USER': u'S-1-5-21-2917001131-1385516553-613696311-1108',
'HTTP_X_USER_DOMAIN_ID': None,
'HTTP_X_AUTH_PROJECT_ID': '69f5cff441e04554b285d7772630dec1',
'HTTP_X_DOMAIN_NAME': None,
'HTTP_X_PROJECT_NAME': None,
'HTTP_X_PROJECT_ID': None,
'HTTP_X_USER_NAME': u'S-1-5-21-2917001131-1385516553-613696311-1108'

Comparing the interaction of nova-api-os-compute with keystone for the
token validation between an internal user and a federated user, the
following is seen:

### federated user ###
2015-07-03 14:43:05.229 8103 DEBUG keystoneclient.session [-] REQ: curl -g -i --insecure -X GET https://sp.testenvironment.local:5000/v3/auth/tokens -H "X-Subject-Token: {SHA1}acff9b5962270fec270e693eacb4c987c335f5c5" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a6a8a70ae39c533379eccd51b6d253f264d59f14" _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:193
2015-07-03 14:43:05.265 8103 DEBUG keystoneclient.session [-] RESP: [200] content-length: 402 x-subject-token: {SHA1}acff9b5962270fec270e693eacb4c987c335f5c5 vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 03 Jul 2015 14:43:05 GMT content-type: application/json x-openstack-request-id: req-df3dce71-3174-4753-b883-11eb31a67d7c
RESP BODY: {"token": {"methods": ["token"], "expires_at": "2015-07-04T02:43:04.000000Z", "extras": {}, "user": {"OS-FEDERATION": {"identity_provider": {"id": "adfs-idp"}, "protocol": {"id": "saml2"}, "groups": []}, "id": "S-1-5-21-2917001131-1385516553-613696311-1108", "name": "S-1-5-21-2917001131-1385516553-613696311-1108"}, "audit_ids": ["_a6BbQ6mSoGAY2u9NN0tFA"], "issued_at": "2015-07-03T14:43:04.000000Z"}}
 
### internal user ###
2015-07-03 14:28:31.875 8103 DEBUG keystoneclient.session [-] REQ: curl -g -i --insecure -X GET https://sp.testenvironment.local:5000/v3/auth/tokens -H "X-Subject-Token: {SHA1}b9c6748d65a0492faa9862fabf0a56fd5fdd255d" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a6a8a70ae39c533379eccd51b6d253f264d59f14" _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:193
2015-07-03 14:28:31.949 8103 DEBUG keystoneclient.session [-] RESP: [200] content-length: 6691 x-subject-token: {SHA1}b9c6748d65a0492faa9862fabf0a56fd5fdd255d vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 03 Jul 2015 14:28:31 GMT content-type: application/json x-openstack-request-id: req-6e0ed9f4-46c3-4c79-b444-f72963fc9503
RESP BODY: {"token": {"methods": ["password"], "roles": [{"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "expires_at": "2015-07-04T02:28:31.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "0f491c8551c04cdc804a479af0bf13ec", "name": "demo"}, "catalog": "<removed>", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "76c8c3017c954d88a6ad69ee4cb656d6", "name": "test"}, "audit_ids": ["aAN_V0c6SLSI0Rm1hoScCg"], "issued_at": "2015-07-03T14:28:31.000000Z"}}

The data structures that come back from keystone are clearly quite
different.

### configuration environment ###

Ubuntu 14.04 OS
nova==12.0.0.0a1.dev51 # commit a4f4be370be06cfc9aa3ed30d2445277e832376f from master branch
keystone==8.0.0.0a1.dev12 # commit a7ca13b687dd284f0980d768b11a3d1b52b4106e from master branch
python-keystoneclient==1.6.1.dev19 # commit d238cc9af4927d1092de207db978536d712af129 from master branch
python-openstackclient==1.5.1.dev11# commit 2d6bc8f4c38dbf997e3e71119f13f0328b4a8669 from master branch
python-novaclient==2.26.1.dev25 # commit 3c2ff0faad8c84777ffe7d9946a1bc4486116084 from master branch
keystonemiddleware==2.0.0
oslo.concurrency==2.1.0
oslo.config==1.12.1
oslo.context==0.4.0
oslo.db==1.12.0
oslo.i18n==2.0.0
oslo.log==1.5.0
oslo.messaging==1.15.0
oslo.middleware==2.3.0
oslo.policy==0.6.0
oslo.serialization==1.6.0
oslo.utils==1.6.0

Keystone is configured as a Shibboleth SP with a trust relationship
between it and an ADFS IdP.

The mapping rules are setup as follows - note that the user's
default_project_id was added in an attempt to see whether it helped. It
does seem to reflect in the HTTP_X_AUTH_PROJECT_ID at the nova api as
shown above.

[
  {
    "local": [
      {
        "user": {
          "id": "{0}",
          "name": "{1}_{2}",
          "email": "{3}",
          "default_project_id": "69f5cff441e04554b285d7772630dec1"
        }
      }
    ],
    "remote": [
      {
        "type": "primarysid"
      },
      {
        "type": "givenname"
      },
      {
        "type": "surname"
      },
      {
        "type": "upn"
      }
    ]
  },
  {
    "local": [
      {
        "group": {
          "name": "fedgroup",
          "domain": { "name": "Default" }
        }
      }
    ],
    "remote": [
      {
        "type": "Group",
        "any_one_of": [
          "Domain Users"
        ]
      }
    ]
  }
]

A Shibboleth session shows the following attributes available from the
IdP:

Attributes
Group: Domain Users
emailaddress: superego@testenvironment.local
givenname: Super
primarysid: S-1-5-21-2917001131-1385516553-613696311-1108
surname: Ego
upn: superego@testenvironment.local

Role, project, group lists are available here:
http://pastebin.com/gcnr9bbT

** Affects: keystone
     Importance: Undecided
     Assignee: Marek Denis (marek-denis)
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1471289

Title:
  Fernet tokens and Federated Identities result in token scope failures

Status in OpenStack Identity (Keystone):
  New

Bug description:
  When keystone is configured to use fernet tokens and also configured
  to be a SP for an external IDP then the token data received by nova
  and other services appear to not contain the right information,
  resulting in errors from nova-api-os-compute such as:

  Returning 400 to user: Malformed request URL: URL's project_id
  '69f5cff441e04554b285d7772630dec1' doesn't match Context's project_id
  'None'

  When keystone is switched to use uuid tokens, then everything works as
  expected.

  Further debugging of the request to the nova api shows:

  'HTTP_X_USER_DOMAIN_NAME': None,
  'HTTP_X_DOMAIN_ID': None,
  'HTTP_X_PROJECT_DOMAIN_ID': None,
  'HTTP_X_ROLES': '',
  'HTTP_X_TENANT_ID': None,
  'HTTP_X_PROJECT_DOMAIN_NAME': None,
  'HTTP_X_TENANT': None,
  'HTTP_X_USER': u'S-1-5-21-2917001131-1385516553-613696311-1108',
  'HTTP_X_USER_DOMAIN_ID': None,
  'HTTP_X_AUTH_PROJECT_ID': '69f5cff441e04554b285d7772630dec1',
  'HTTP_X_DOMAIN_NAME': None,
  'HTTP_X_PROJECT_NAME': None,
  'HTTP_X_PROJECT_ID': None,
  'HTTP_X_USER_NAME': u'S-1-5-21-2917001131-1385516553-613696311-1108'

  Comparing the interaction of nova-api-os-compute with keystone for the
  token validation between an internal user and a federated user, the
  following is seen:

  ### federated user ###
  2015-07-03 14:43:05.229 8103 DEBUG keystoneclient.session [-] REQ: curl -g -i --insecure -X GET https://sp.testenvironment.local:5000/v3/auth/tokens -H "X-Subject-Token: {SHA1}acff9b5962270fec270e693eacb4c987c335f5c5" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a6a8a70ae39c533379eccd51b6d253f264d59f14" _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:193
  2015-07-03 14:43:05.265 8103 DEBUG keystoneclient.session [-] RESP: [200] content-length: 402 x-subject-token: {SHA1}acff9b5962270fec270e693eacb4c987c335f5c5 vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 03 Jul 2015 14:43:05 GMT content-type: application/json x-openstack-request-id: req-df3dce71-3174-4753-b883-11eb31a67d7c
  RESP BODY: {"token": {"methods": ["token"], "expires_at": "2015-07-04T02:43:04.000000Z", "extras": {}, "user": {"OS-FEDERATION": {"identity_provider": {"id": "adfs-idp"}, "protocol": {"id": "saml2"}, "groups": []}, "id": "S-1-5-21-2917001131-1385516553-613696311-1108", "name": "S-1-5-21-2917001131-1385516553-613696311-1108"}, "audit_ids": ["_a6BbQ6mSoGAY2u9NN0tFA"], "issued_at": "2015-07-03T14:43:04.000000Z"}}
   
  ### internal user ###
  2015-07-03 14:28:31.875 8103 DEBUG keystoneclient.session [-] REQ: curl -g -i --insecure -X GET https://sp.testenvironment.local:5000/v3/auth/tokens -H "X-Subject-Token: {SHA1}b9c6748d65a0492faa9862fabf0a56fd5fdd255d" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}a6a8a70ae39c533379eccd51b6d253f264d59f14" _http_log_request /usr/local/lib/python2.7/dist-packages/keystoneclient/session.py:193
  2015-07-03 14:28:31.949 8103 DEBUG keystoneclient.session [-] RESP: [200] content-length: 6691 x-subject-token: {SHA1}b9c6748d65a0492faa9862fabf0a56fd5fdd255d vary: X-Auth-Token keep-alive: timeout=5, max=100 server: Apache/2.4.7 (Ubuntu) connection: Keep-Alive date: Fri, 03 Jul 2015 14:28:31 GMT content-type: application/json x-openstack-request-id: req-6e0ed9f4-46c3-4c79-b444-f72963fc9503
  RESP BODY: {"token": {"methods": ["password"], "roles": [{"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "expires_at": "2015-07-04T02:28:31.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "0f491c8551c04cdc804a479af0bf13ec", "name": "demo"}, "catalog": "<removed>", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "76c8c3017c954d88a6ad69ee4cb656d6", "name": "test"}, "audit_ids": ["aAN_V0c6SLSI0Rm1hoScCg"], "issued_at": "2015-07-03T14:28:31.000000Z"}}

  The data structures that come back from keystone are clearly quite
  different.

  ### configuration environment ###

  Ubuntu 14.04 OS
  nova==12.0.0.0a1.dev51 # commit a4f4be370be06cfc9aa3ed30d2445277e832376f from master branch
  keystone==8.0.0.0a1.dev12 # commit a7ca13b687dd284f0980d768b11a3d1b52b4106e from master branch
  python-keystoneclient==1.6.1.dev19 # commit d238cc9af4927d1092de207db978536d712af129 from master branch
  python-openstackclient==1.5.1.dev11# commit 2d6bc8f4c38dbf997e3e71119f13f0328b4a8669 from master branch
  python-novaclient==2.26.1.dev25 # commit 3c2ff0faad8c84777ffe7d9946a1bc4486116084 from master branch
  keystonemiddleware==2.0.0
  oslo.concurrency==2.1.0
  oslo.config==1.12.1
  oslo.context==0.4.0
  oslo.db==1.12.0
  oslo.i18n==2.0.0
  oslo.log==1.5.0
  oslo.messaging==1.15.0
  oslo.middleware==2.3.0
  oslo.policy==0.6.0
  oslo.serialization==1.6.0
  oslo.utils==1.6.0

  Keystone is configured as a Shibboleth SP with a trust relationship
  between it and an ADFS IdP.

  The mapping rules are setup as follows - note that the user's
  default_project_id was added in an attempt to see whether it helped.
  It does seem to reflect in the HTTP_X_AUTH_PROJECT_ID at the nova api
  as shown above.

  [
    {
      "local": [
        {
          "user": {
            "id": "{0}",
            "name": "{1}_{2}",
            "email": "{3}",
            "default_project_id": "69f5cff441e04554b285d7772630dec1"
          }
        }
      ],
      "remote": [
        {
          "type": "primarysid"
        },
        {
          "type": "givenname"
        },
        {
          "type": "surname"
        },
        {
          "type": "upn"
        }
      ]
    },
    {
      "local": [
        {
          "group": {
            "name": "fedgroup",
            "domain": { "name": "Default" }
          }
        }
      ],
      "remote": [
        {
          "type": "Group",
          "any_one_of": [
            "Domain Users"
          ]
        }
      ]
    }
  ]

  A Shibboleth session shows the following attributes available from the
  IdP:

  Attributes
  Group: Domain Users
  emailaddress: superego@testenvironment.local
  givenname: Super
  primarysid: S-1-5-21-2917001131-1385516553-613696311-1108
  surname: Ego
  upn: superego@testenvironment.local

  Role, project, group lists are available here:
  http://pastebin.com/gcnr9bbT

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1471289/+subscriptions

Follow ups

References

  Thread Previous • Date Previous • • Thread Next