yahoo-eng-team team mailing list archive

[OpenStack Infra <1813085@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/633288
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=af3aef940c0162f12752a65282368d16c2d17c4f
Submitter: Zuul
Branch:    master

commit af3aef940c0162f12752a65282368d16c2d17c4f
Author: Lance Bragstad <lbragstad@xxxxxxxxx>
Date:   Fri Jan 25 20:55:15 2019 +0000

    Handle special cases with msgpack and python3
    
    We attempt to be clever about string types in the token formatters.
    We do this because in some cases, not all items in a token payload
    are serialized to byte strings. To add flexibility for this, we use
    tuples with a boolean value that denotes if the accompanying value is
    a byte string or not. This helps us safely re-inflate the value from
    a byte string back to it's .hex representations, typically with UUID
    strings.
    
    With python3, we actually hit an interesting case where what we pass
    into the token payload doesn't actually maintain that state due to the
    usage of msgpack. The msgpack library returns byte strings even though
    the initial value may not have been a byte string. This breaks the
    logic we have for the associated boolean value because the string type
    changes and the boolean does not.
    
    This commit adds a couple of if/statements to detect if we running on
    python3 and if the boolean mismatches the actual value type. Then, it
    attempts to do the right thing by decoding the string.
    
    We should think about how we want to do this, or if there is a better
    way.
    
    Change-Id: Iaecd45ef985cbf5ff4a6a724df96c1304a927247
    Closes-Bug: 1813085


** Changed in: keystone
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1813085

Title:
  domain_id 'default' is not decoded from bytes with federated scoped
  tokens

Status in OpenStack Identity (keystone):
  Fix Released

Bug description:
  When attempting to make calls to the Keystone API, requests using a
  scoped federated token fail with a message like the following:

      {
          "error": {
              "code": 404,
              "message": "Could not find domain: b'default'.",
              "title": "Not Found"
          }
      }

  
  To reproduce:
   1. get an unscoped token via the federated auth endpoint, e.g. /v3/OS-FEDERATION/identity_providers/myidp/protocols/mapped/auth
   2. request an unscoped token against the default domain, e.g.:

      $ http post https://example.com/keystone/v3/auth/tokens << EOF
      {
          "auth": {
              "identity": {
                  "methods": [
                      "token"
                  ],
                  "token": {
                      "id": "$token"
                  }
              },
              "scope": {
                  "domain": {
                      "id": "default"
                  }
              }
          }
      }
      EOF
   3. Attempt to get your own user data, e.g. 

      $ http get https://example.com/keystone/v3/users/$user x-auth-token:"$scoped_token"
      HTTP/1.1 404 Not Found
      Connection: keep-alive
      Content-Length: 95
      Content-Type: application/json
      Date: Wed, 23 Jan 2019 20:46:12 GMT
      Server: nginx/1.13.12
      Strict-Transport-Security: max-age=15724800; includeSubDomains
      Vary: X-Auth-Token
      x-openstack-request-id: req-foo

      {
          "error": {
              "code": 404,
              "message": "Could not find domain: b'default'.",
              "title": "Not Found"
          }
      }

  The expected result looks like this (with a patch applied to decode
  the domain_id to a str):

      $ http get https://example.com/keystone/v3/users/$user x-auth-token:"$scoped_token"                                                               
      HTTP/1.1 200 OK
      Connection: keep-alive
      Content-Encoding: gzip
      Content-Type: application/json
      Date: Wed, 23 Jan 2019 21:45:11 GMT
      Server: nginx/1.13.12
      Strict-Transport-Security: max-age=15724800; includeSubDomains
      Transfer-Encoding: chunked
      Vary: Accept-Encoding
      Vary: X-Auth-Token
      x-openstack-request-id: req-bar

      {
          "user": {
              "domain_id": "default",
              "email": "foo@xxxxxxxxxxx",
              "enabled": true,
              "id": "7b3bbc3252c44f139eb8a609eccc299b",
              "links": {
                  "self": "https://example.com/keystone/v3/users/7b3bbc3252c44f139eb8a609eccc299b";
              },
              "name": "foo@xxxxxxxxxxx",
              "options": {},
              "password_expires_at": null
          }
      }

  After digging through the code, I think the root cause may be that in
  FederatedScopedPayload.disassemble() [1], scope_id is never decoded
  like it is in DomainScopedPayload.disassemble() [2]. The bytes value
  eventually makes its way down to Manager.get_domain() [3] where the
  check fails because 'default' != b'default'.

  This was all tested against keystone 14.0.2.dev7 (latest rocky
  release) running python 3.6.7 with uwsgi 2.0.17.

  [1] https://github.com/openstack/keystone/blob/3db38cabcbff305693ae1638ad63208701926bd0/keystone/token/token_formatters.py#L605-L633
  [2] https://github.com/openstack/keystone/blob/3db38cabcbff305693ae1638ad63208701926bd0/keystone/token/token_formatters.py#L415-L424
  [3] https://github.com/openstack/keystone/blob/3db38cabcbff305693ae1638ad63208701926bd0/keystone/resource/core.py#L699-L700
  [4] https://github.com/openstack/keystone/blob/3db38cabcbff305693ae1638ad63208701926bd0/keystone/token/token_formatters.py#L195-L196

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1813085/+subscriptions