← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #79361

[Bug 1830782] Re: stein: openstack-dashboard gui not showing newly created project/users under newly domain

  Thread Previous • Date Previous • • Thread Next 
On further debugging, it appears that there is an issue in horizon
(caused due to a change in keystone) with the use of a scoped tokens for
the admin user when multi-domain is enabled.

The scenario is as follows:

1. Multi domain is enabled.
2. The admin user is logged in with credentials using an admin domain.
3. Domain context is set to a domain in which the admin user is not a member.
4. The admin user attempts to list the projects or users.
5. A domain scoped token is used by horizon to list the projects, due to the code in [1]
6. No users are returned from keystone because, due to change [2] the users are filtered as the token contains the admin domain, not the target domain or users to list.

It's quite involved!

I'm not sure if the issue is:

1. Keystone shouldn't be filtering this list.
2. Horizon shouldn't be using a domain scoped token for the admin user (e.g. the openstack CLI doesn't use a domain scoped token to list the users in the domain, or an admin user).
3. Something else.

Horizon appears to only start using the domain scoped token after the
domain context is set.  Also, it only appears (in my testing) to use it
for the user list and (maybe) project list -- I focussed on the user
list.  It looks like a new token is requested to perform the user list
and that this one is domain scoped.

I can do further testing as necessary.


References:
[1] Horizon, openstack_dashboard/api/keystone.py (def keystoneclient:) https://github.com/openstack/horizon/blob/stable/stein/openstack_dashboard/api/keystone.py#L167
[2] Keystone, change Id: I60b2e2b8af172c369eab0eb2c29f056f5c98ad16, https://review.opendev.org/#/c/647587/ (for user list)


Debugging info:

I added some debug LOG lines to the various bits of horizon and keystone
to try to work out what was going on.  The following is a comparison
between Horizon and the OpenStack CLI in listing users for a domain
"test-domain":

Preamble: The test set up:

The test is listing users for the "test-domain" on the OpenStack CLI and
using the Horizon dashboard.


Domain list:

+----------------------------------+----------------+---------+-----------------+
| ID                               | Name           | Enabled | Description     |
+----------------------------------+----------------+---------+-----------------+
| 4c97d83fd8f34507aa5849710218272e | default        | True    | Created by Juju |
| 917f251e6fc24c389f1e3f3624d701d1 | admin_domain   | True    | Created by Juju |
| be5450b76a2348c48df0d0571295de40 | test-domain2   | True    |                 |
| c9ca71bd88894017a6b6448dfcffeb68 | test-domain    | True    |                 |
| ecb1e99a62534253a5b515dcfc218733 | service_domain | True    | Created by Juju |
+----------------------------------+----------------+---------+-----------------+

The "admin" user is in the admin_domain.

Project List:

+----------------------------------+---------------+
| ID                               | Name          |
+----------------------------------+---------------+
| 1014c1815147453b8bd77de578467a80 | demo          |
| 49ae284fd4aa42208573d9c399a95eee | services      |
| 7581c43d252848dface4c75e2b921224 | test-project  |
| 75c183f2aece43e2860be59926e244fb | admin         |
| 9bc98ed16a7547e0b11d002172ab1d6e | test-project2 |
| 9c619796ef91470bba2d30427bd7adc6 | admin         |
| a7c8c2f4d11844619fb22753ab4d7a80 | services      |
| b8eb986468684e7ab4c7eb92542d3e58 | alt_demo      |
+----------------------------------+---------------+

The "admin" users is in the "admin" project.

openstack user list
+----------------------------------+----------+
| ID                               | Name     |
+----------------------------------+----------+
| 8973385dd5ca467fb4be7a3eca7a603f | admin    |
| 8aeaead88fdc49c6a44a3983d3ff8c63 | demo     |
| b7beaf7d43b144d5b71acb33f0abb87d | alt_demo |
+----------------------------------+----------+

+----------------------------------+------------------+
| ID                               | Name             |
+----------------------------------+------------------+
| 9c1fa58637a64cd387922a4b2b8ce522 | test-domain-user |
+----------------------------------+------------------+


---

OpenStack CLI debug for "openstack user list --domain=test-domain"

OS_VARS:

OS_AUTH_URL=http://10.5.0.56:5000/v3
OS_DOMAIN_NAME=admin_domain
OS_REGION_NAME=RegionOne
OS_PROJECT_NAME=admin
OS_PROJECT_DOMAIN_NAME=admin_domain
OS_USER_DOMAIN_NAME=admin_domain
OS_AUTH_VERSION=3
OS_IDENTITY_API_VERSION=3
OS_PASSWORD=openstack
OS_USERNAME=admin


Token:
(keystone.token.provider): 2019-07-17 18:03:04,001 DEBUG ZZZZ - the token: gAAAAABdL2LX_HT3mi4RO0KcwuqYaJ-NoY-gDMQtcKm-QDDJ0o-SsiH1BOaI5LlhbPLVyiKw7amvGcuwwuM9LLCCBb0VGkyIs2cmkTlHAC
rOyXvtHAdIcRTwzOVmdQ3wsswwB02jnRL2c49w4a9dfii1eMUhxwtCs-ZDkxE8k52Yf9lkXDnDyzQ contains:
(keystone.token.provider): 2019-07-17 18:03:04,001 DEBUG domain: None, domain_scoped: False, user: {'email': 'juju@localhost', 'id': '8973385dd5ca467fb4be7a3eca7a603f', 'name': 'adm
in', 'domain_id': '917f251e6fc24c389f1e3f3624d701d1', 'enabled': True, 'password_expires_at': None, 'options': {}}

On Keystone using log_foreisic:

+20164:5d2f62d7:17|GET /v3/domains/test-domain HTTP/1.1|Host:10.5.0.56%3a35357|User-Agent:python-keystoneclient|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2LX_HT3mi4RO0KcwuqYaJ-NoY-gDMQtcKm-QDDJ0o-SsiH1BOaI5LlhbPLVyiKw7amvGcuwwuM9LLCCBb0VGkyIs2cmkTlHACrOyXvtHAdIcRTwzOVmdQ3wsswwB02jnRL2c49w4a9dfii1eMUhxwtCs-ZDkxE8k52Yf9lkXDnDyzQ
-20164:5d2f62d7:17
+20164:5d2f62d7:18|GET /v3/domains?name=test-domain HTTP/1.1|Host:10.5.0.56%3a35357|User-Agent:python-keystoneclient|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2LX_HT3mi4RO0KcwuqYaJ-NoY-gDMQtcKm-QDDJ0o-SsiH1BOaI5LlhbPLVyiKw7amvGcuwwuM9LLCCBb0VGkyIs2cmkTlHACrOyXvtHAdIcRTwzOVmdQ3wsswwB02jnRL2c49w4a9dfii1eMUhxwtCs-ZDkxE8k52Yf9lkXDnDyzQ
-20164:5d2f62d7:18
+20164:5d2f62d7:19|GET /v3/users?domain_id=c9ca71bd88894017a6b6448dfcffeb68 HTTP/1.1|Host:10.5.0.56%3a35357|User-Agent:python-keystoneclient|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2LX_HT3mi4RO0KcwuqYaJ-NoY-gDMQtcKm-QDDJ0o-SsiH1BOaI5LlhbPLVyiKw7amvGcuwwuM9LLCCBb0VGkyIs2cmkTlHACrOyXvtHAdIcRTwzOVmdQ3wsswwB02jnRL2c49w4a9dfii1eMUhxwtCs-ZDkxE8k52Yf9lkXDnDyzQ
-20164:5d2f62d7:19

I.e. the OpenStack CLI used a non-domain scoped token to access the
/v3/users?domain_id=... endpoint


---

In Horizon/dashboard:

(listing users for test-domain domain, with user "admin" logged in with
"admin_domain" and a password):


Token:
(keystone.token.provider): 2019-07-17 17:52:15,538 DEBUG ZZZZ - the token: gAAAAABdL2BMm4RFbxzvoHI897QYkhLQLvfdAMKN25pThs5XCMLrpQRm9VmjXYvh52MAiK_6XUKRyBDBC9OIlJPbEPGU981kRIdwp5T8e1bby4UJeerBKoEB1BQ69c8T23HsoGsN0Kwr4iWST7ECXKfep6k7-C3TsZU74xmsflTlkA1DyDCPGZSN1g_JiPyfVyPTqa96AzMN contains:
(keystone.token.provider): 2019-07-17 17:52:15,539 DEBUG domain: {'id': '917f251e6fc24c389f1e3f3624d701d1', 'name': 'admin_domain', 'description': 'Created by Juju', 'enabled': True, 'tags': []}, domain_scoped: True, user: {'email': 'juju@localhost', 'id': '8973385dd5ca467fb4be7a3eca7a603f', 'name': 'admin', 'domain_id': '917f251e6fc24c389f1e3f3624d701d1', 'enabled': True, 'password_expires_at': None, 'options': {}}

(note that it is domain_scoped)

Notice that the first two queries use the domain_scoped token:

+20165:5d2f6184:17|GET /v3/users?domain_id=c9ca71bd88894017a6b6448dfcffeb68 HTTP/1.1|Host:10.5.0.56%3a35357|User-Agent:python-keystoneclient|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2BMm4RFbxzvoHI897QYkhLQLvfdAMKN25pThs5XCMLrpQRm9VmjXYvh52MAiK_6XUKRyBDBC9OIlJPbEPGU981kRIdwp5T8e1bby4UJeerBKoEB1BQ69c8T23HsoGsN0Kwr4iWST7ECXKfep6k7-C3TsZU74xmsflTlkA1DyDCPGZSN1g_JiPyfVyPTqa96AzMN|Forwarded:for=10.5.0.105;by=python-keystoneclient
-20165:5d2f6184:17
+20164:5d2f6184:d|GET /v3/domains HTTP/1.1|Host:10.5.0.56%3a35357|User-Agent:python-keystoneclient|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2BMm4RFbxzvoHI897QYkhLQLvfdAMKN25pThs5XCMLrpQRm9VmjXYvh52MAiK_6XUKRyBDBC9OIlJPbEPGU981kRIdwp5T8e1bby4UJeerBKoEB1BQ69c8T23HsoGsN0Kwr4iWST7ECXKfep6k7-C3TsZU74xmsflTlkA1DyDCPGZSN1g_JiPyfVyPTqa96AzMN|Forwarded:for=10.5.0.105;by=python-keystoneclient
-20164:5d2f6184:d


(but then next is using a different token)

+20164:5d2f6184:e|GET
/v3/users/8973385dd5ca467fb4be7a3eca7a603f/projects
HTTP/1.1|Host:10.5.0.56%3a5000|User-Agent:python-keystoneclient|Accept-
Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X
-Auth-Token:gAAAAABdL2BKK3ew3ilF-
JOVVZbnVDLRxLmS3VlJTmJY8ENYFrv4lqdcJf2LALE1BvcG5D5ay_1Ce1NJVdr4CTr7haHa0oP8_bcJ7qWMxnYlMEPL-
Niw-8ddzlNHdcFtR45dOJgk0snGr1biueO_kEG9O04U_fXQhQ


-20164:5d2f6184:e
+20164:5d2f6184:f|GET /v3 HTTP/1.1|Host:10.5.0.56%3a5000|User-Agent:python-novaclient keystoneauth1/3.13.1 python-requests/2.18.4 CPython/3.6.8|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive
-20164:5d2f6184:f

Then a token request is made:

+20164:5d2f6184:10|POST /v3/auth/tokens HTTP/1.1|Host:10.5.0.56%3a5000|User-Agent:python-novaclient keystoneauth1/3.13.1 python-requests/2.18.4 CPython/3.6.8|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2BPSlVP8HSWBoxcd0u9j2Z1vX5wz2dYvvpFjyq2ixvsVYpC0xpet2vMjE9l6FNrCv4yiYCmww6d4QUSLrna7bSNGc4fkEAIWa-RdJ9fVkC40P3j1litDkCx0aQ2WhlOYUFTa1-Sh7vr0QckzHeuwd3LviSRLKhCi55NccgCW55RCDchW43l-RCLNsJKBmc5W00C|Content-Type:application/json|Content-Length:337
-20164:5d2f6184:10

+20164:5d2f6184:11|GET /v3 HTTP/1.1|Host:10.5.0.56%3a5000|User-Agent:openstack_dashboard keystoneauth1/3.13.1 python-requests/2.18.4 CPython/3.6.8|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|Forwarded:for=10.5.0.105;by=openstack_dashboard keystoneauth1/3.13.1 python-requests/2.18.4 CPython/3.6.8
-20164:5d2f6184:11

And the projects for that "admin" user are listed with that token.

+20165:5d2f6185:18|GET /v3/users/8973385dd5ca467fb4be7a3eca7a603f/projects HTTP/1.1|Host:10.5.0.56%3a5000|User-Agent:python-keystoneclient|Accept-Encoding:gzip, deflate|Accept:application/json|Connection:keep-alive|X-Auth-Token:gAAAAABdL2BKK3ew3ilF-JOVVZbnVDLRxLmS3VlJTmJY8ENYFrv4lqdcJf2LALE1BvcG5D5ay_1Ce1NJVdr4CTr7haHa0oP8_bcJ7qWMxnYlMEPL-Niw-8ddzlNHdcFtR45dOJgk0snGr1biueO_kEG9O04U_fXQhQ
-20165:5d2f6185:18


** Also affects: horizon
   Importance: Undecided
       Status: New

** Changed in: charm-keystone
       Status: Confirmed => Invalid

** Changed in: charm-openstack-dashboard
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1830782

Title:
  stein: openstack-dashboard gui not showing  project/users for a
  selected domain (via "set domain context) for the admin user.

Status in OpenStack keystone charm:
  Invalid
Status in OpenStack openstack-dashboard charm:
  Invalid
Status in OpenStack Dashboard (Horizon):
  New
Status in OpenStack Bundles:
  Invalid

Bug description:
  OpenStack-dashboard gui not showing newly created project/users under
  a new domain, but everything shows fine in CLI. The project and users
  were created on the horizon gui a notification in green shows
  successfully but nothing shows in the gui. The OpenStack Stein version
  (openstack telemetry #53 ) was installed with juju charm bundle
  running 4 nodes any idea where to look? I'm wondering how the Horizon
  gui get updated (who's responsible to update the horizon gui) I would
  appreciate your help Thanks.

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-keystone/+bug/1830782/+subscriptions
  Thread Previous • Date Previous • • Thread Next