yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1828126] [NEW] [<= Queens] With token-provider='uuid', roles of dynamically obtained federated groups are not taken into account during token-based authentication (for project-scoped token creation)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Dmitrii Shcherbakov <1828126@xxxxxxxxxxxxxxxxxx>
Date: Tue, 07 May 2019 23:36:24 -0000
Reply-to: Bug 1828126 <1828126@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

[Overview]
The relevant part of the federated authentication process after the IdP and SP token parsing stages is as follows:

1) WSGI environment variables created based on token attributes (e.g. SAML token attributes) are passed down to Keystone;
2) Keystone creates a shadow mapped user in the db and tries to map token attributes to objects such as groups, roles and projects in the DB based on a custom mapping created by an operator;
3) groups that may be obtained from token attributes are matched against groups in Keystone but the user is not included into those groups in Keystone DB (to support dynamic group membership changes at the IdP side). If any of the target groups do not exist in Keystone authentication fails;
4) A domain-scoped federated token is created (e.g. by Horizon) and then a project-scoped token is created using the previous token as the authentication method.

(4) is where the problem occurs.
 
[Environment]
Queens, 19.04 charms, token-provider='uuid' for charm-keystone.

openstack commands used to configure an IdP:
https://paste.ubuntu.com/p/nj6MdQDKk2/

keystone.conf sections:
[auth]
methods = external,password,token,oauth1,totp,application_credential,saml2
[federation]
trusted_dashboard = https://dashboard.maas/auth/websso/
[saml2]
remote_id_attribute = MELLON_IDP

IdP is ADFS in this case which uses a windows account name as NAMEID and adds an attribute which corresponds to a group ID (the group name in Active Directory is the OpenStack group ID). The resulting SAML token then contains the following elements:
 <AttributeStatement> <Attribute Name="groups"> <AttributeValue>3f031869ef9f4dc49a342d6be69e98b3</AttributeValue> </Attribute> </AttributeStatement>

The direct usage of a group ID is present to rule out group name to ID
resolution problems.

[Use-case]

Automatic project provisioning and Member role assignment to users is
not used on purpose to manage user access to projects via group to role
assignments. A user is a assigned to a group at the IdP side and the
keystone database does not contain any role assignments for shadow-
mapped users. `openstack role assignment list --names` will not contain
anything related to group assignments - all group membership information
will only be exposed in a token.

[Problem Description]

1) the first token (federated, obtained via v3 federation API) is domain-scoped and authentication succeeds for it;
2) then a client (e.g. Horizon) gets a project-scoped token based on that federated token (token authentication & regular v3 API) for which roles need to be populated - including the roles to access the target project;
3) the roles for the second token are not populated correctly based on the (dynamic) group assignments that came from the SAML token for the first token - clearly the role population code-path for the second token is not aware of groups that came dynamically with the SAML token. The expected result would be awareness of groups assigned to the shadow mapped user and then inference of roles from groups based on group to role assignments in the Keystone DB. This explains the fact that project auto-provisioning and project role assignment to shadow users directly works properly (because this can be queried by keystone from its db).

The visible end-result for a user authenticating via the dashboard is
represented in a form of errors such as "Unauthorized: Unable to..." for
any accessed dashboard pane.

[Symptoms]

Example: https://paste.ubuntu.com/p/syxxWmdyD7/
(keystone.token.provider): 2019-05-07 17:47:01,947 DEBUG Unable to validate token: The request you have made requires authentication.

Project-scoped token example (contains the right group and "methods": ["token", "saml2"]) as queried directly from the db:
https://paste.ubuntu.com/p/rRgXSctgWT/

rpdb trace - first pass at finding where it fails:
https://paste.ubuntu.com/p/DhG4HXCnBB/

Second pass (the most useful) - a trace point in keystone/token/providers/common.py get_token_data() going down to keystone/token/providers/common.py(432)_populate_roles() where the Unauthorized exception is thrown:
https://paste.ubuntu.com/p/pjRf7qBzcX/


[Root Cause]

Based on the symptoms it is clear that _populate_roles (unlike
populate_roles_for_federated_user) does not include group roles for
groups obtained via federated authentication:

https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L408-L432
(_populate_roles)

https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L168-L193
(_get_roles_for_user, has a branch to work with group roles but for
system-scoped tokens only)

https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L190-L193
(get_roles_for_user_and_project gets user to role assignments which are
not present in this case)

Which in the end leads to exception.Unauthorized being thrown by
Keystone
https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L416-L432

[Mitigation]

Usage of fernet tokens for which the same behavior does not occur.

For existing openstack-charm-deployed environments, running this on a live cloud is the way forward:
juju config keystone token-provider='fernet'

# see the upgrade notes https://specs.openstack.org/openstack/charm-
specs/specs/rocky/implemented/keystone-fernet-tokens.html#upgrading-an-
existing-system

Note: this is more of a documentation bug as UUID tokens were removed in
Rocky and so there is no point in fixing this at the distro level or
upstream.

** Affects: keystone
     Importance: Undecided
         Status: New

** Affects: keystone (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: cpe-onsite

** Attachment added: "reproducer-queens-bionic.png"
   https://bugs.launchpad.net/bugs/1828126/+attachment/5262180/+files/reproducer-queens-bionic.png

** Also affects: keystone
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1828126

Title:
  [<= Queens] With token-provider='uuid', roles of dynamically obtained
  federated groups are not taken into account during token-based
  authentication (for project-scoped token creation)

Status in OpenStack Identity (keystone):
  New
Status in keystone package in Ubuntu:
  New

Bug description:
  [Overview]
  The relevant part of the federated authentication process after the IdP and SP token parsing stages is as follows:

  1) WSGI environment variables created based on token attributes (e.g. SAML token attributes) are passed down to Keystone;
  2) Keystone creates a shadow mapped user in the db and tries to map token attributes to objects such as groups, roles and projects in the DB based on a custom mapping created by an operator;
  3) groups that may be obtained from token attributes are matched against groups in Keystone but the user is not included into those groups in Keystone DB (to support dynamic group membership changes at the IdP side). If any of the target groups do not exist in Keystone authentication fails;
  4) A domain-scoped federated token is created (e.g. by Horizon) and then a project-scoped token is created using the previous token as the authentication method.

  (4) is where the problem occurs.
   
  [Environment]
  Queens, 19.04 charms, token-provider='uuid' for charm-keystone.

  openstack commands used to configure an IdP:
  https://paste.ubuntu.com/p/nj6MdQDKk2/

  keystone.conf sections:
  [auth]
  methods = external,password,token,oauth1,totp,application_credential,saml2
  [federation]
  trusted_dashboard = https://dashboard.maas/auth/websso/
  [saml2]
  remote_id_attribute = MELLON_IDP

  IdP is ADFS in this case which uses a windows account name as NAMEID and adds an attribute which corresponds to a group ID (the group name in Active Directory is the OpenStack group ID). The resulting SAML token then contains the following elements:
   <AttributeStatement> <Attribute Name="groups"> <AttributeValue>3f031869ef9f4dc49a342d6be69e98b3</AttributeValue> </Attribute> </AttributeStatement>

  The direct usage of a group ID is present to rule out group name to ID
  resolution problems.

  [Use-case]

  Automatic project provisioning and Member role assignment to users is
  not used on purpose to manage user access to projects via group to
  role assignments. A user is a assigned to a group at the IdP side and
  the keystone database does not contain any role assignments for
  shadow-mapped users. `openstack role assignment list --names` will not
  contain anything related to group assignments - all group membership
  information will only be exposed in a token.

  [Problem Description]

  1) the first token (federated, obtained via v3 federation API) is domain-scoped and authentication succeeds for it;
  2) then a client (e.g. Horizon) gets a project-scoped token based on that federated token (token authentication & regular v3 API) for which roles need to be populated - including the roles to access the target project;
  3) the roles for the second token are not populated correctly based on the (dynamic) group assignments that came from the SAML token for the first token - clearly the role population code-path for the second token is not aware of groups that came dynamically with the SAML token. The expected result would be awareness of groups assigned to the shadow mapped user and then inference of roles from groups based on group to role assignments in the Keystone DB. This explains the fact that project auto-provisioning and project role assignment to shadow users directly works properly (because this can be queried by keystone from its db).

  The visible end-result for a user authenticating via the dashboard is
  represented in a form of errors such as "Unauthorized: Unable to..."
  for any accessed dashboard pane.

  [Symptoms]

  Example: https://paste.ubuntu.com/p/syxxWmdyD7/
  (keystone.token.provider): 2019-05-07 17:47:01,947 DEBUG Unable to validate token: The request you have made requires authentication.

  Project-scoped token example (contains the right group and "methods": ["token", "saml2"]) as queried directly from the db:
  https://paste.ubuntu.com/p/rRgXSctgWT/

  rpdb trace - first pass at finding where it fails:
  https://paste.ubuntu.com/p/DhG4HXCnBB/

  Second pass (the most useful) - a trace point in keystone/token/providers/common.py get_token_data() going down to keystone/token/providers/common.py(432)_populate_roles() where the Unauthorized exception is thrown:
  https://paste.ubuntu.com/p/pjRf7qBzcX/

  
  [Root Cause]

  Based on the symptoms it is clear that _populate_roles (unlike
  populate_roles_for_federated_user) does not include group roles for
  groups obtained via federated authentication:

  https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L408-L432
  (_populate_roles)

  https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L168-L193
  (_get_roles_for_user, has a branch to work with group roles but for
  system-scoped tokens only)

  https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L190-L193
  (get_roles_for_user_and_project gets user to role assignments which
  are not present in this case)

  Which in the end leads to exception.Unauthorized being thrown by
  Keystone
  https://opendev.org/openstack/keystone/src/branch/stable/queens/keystone/token/providers/common.py#L416-L432

  [Mitigation]

  Usage of fernet tokens for which the same behavior does not occur.

  For existing openstack-charm-deployed environments, running this on a live cloud is the way forward:
  juju config keystone token-provider='fernet'

  # see the upgrade notes https://specs.openstack.org/openstack/charm-
  specs/specs/rocky/implemented/keystone-fernet-tokens.html#upgrading-
  an-existing-system

  Note: this is more of a documentation bug as UUID tokens were removed
  in Rocky and so there is no point in fixing this at the distro level
  or upstream.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1828126/+subscriptions