yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #88804
[Bug 1971592] Re: System scoped token: System->system information->compute service list results in 403
** Changed in: horizon
Status: Invalid => New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1971592
Title:
System scoped token: System->system information->compute service list
results in 403
Status in OpenStack Dashboard (Horizon):
New
Bug description:
Hi all,
We have started to analyze Openstack Yoga a bit and there, one of the major new feature is the activation of scope based token for regular use in nova. While after some long lasting back and forth in configuring our role assignments and policies we could make it work on one of our test environments (Ubuntu) via Openstack SDK, however, we are still struggling with some system scoped API calls to nova from horizon.
We have an admin user for the domain 'Default' who has set the role ‚admin' for 'system all':
+-------+---------------+-------+---------+--------+--------+-----------+
| Role | User | Group | Project | Domain | System | Inherited |
+-------+---------------+-------+---------+--------+--------+-----------+
| admin | admin@Default | | | | all | False |
+-------+---------------+-------+---------+--------+--------+-----------+
To make login in horizon successful, the user admin is also admin for the domain 'default' and for the project 'admin' in the domain 'default'.
We have configured in local_settings.py:
SYSTEM_SCOPE_SERVICES = ['compute', 'image', 'volume', 'network‘]
(Note: this config line has been reverse engineered from horizon
source code as the syntax is nowhere possible to be found in the docs
yet … - so: not sure if it is correct)
Policy files are identical for horizon as for the services. Policy for this api request is
"os_compute_api:os-services:list": "rule:context_is_admin"
For the user admin, we then get an additional field in the
domain/project top line menu adding a ‚system scope‘ switch (this is
what we understand how it should look like) and - when switching to
system scope - also a 'system' menu in the sidebar becomes visible
(also as expected).
If we then go to System->Systeminformation to see the nova service
list, we get an error ‚Unable to get nova services list‘, given reason
is an error: 'Policy doesn’t allow os_compute_api:os-services:list to
be performed (HTTP 403)‘. Informations for network and volume services
are shown normally (here scoped tokens are not activated yet).
apache2 error log in debug mode results for this call in:
[Wed May 04 13:40:45.314012 2022] [wsgi:error] [pid 3007771:tid 140637774444288] [remote 192.168.1.70:59546] DEBUG:keystoneauth.identity.v3.base:Making authentication request to https://controller:5000/v3/auth/tokens
[Wed May 04 13:40:45.396814 2022] [wsgi:error] [pid 3007771:tid 140637774444288] [remote 192.168.1.70:59546] DEBUG:urllib3.connectionpool:https://controller:5000 "POST /v3/auth/tokens HTTP/1.1" 201 12010
[Wed May 04 13:40:45.397786 2022] [wsgi:error] [pid 3007771:tid 140637774444288] [remote 192.168.1.70:59546] DEBUG:keystoneauth.identity.v3.base:{"token": {"methods": ["password", "token"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "a837078533b64bac8be7f9fb0c57ead6", "name": "admin", "password_expires_at": null}, "audit_ids": ["EAXc26zRR4KUEwDSLwKe4w", "iUCwAVveTAay4FfkHwZ1sA"], "expires_at": "2022-05-04T12:40:30.000000Z", "issued_at": "2022-05-04T11:40:45.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "6f79eb4041a142dfaffb5d1f8bff906c", "name": "admin"}, "is_domain": false, "roles": [{"id": "9109437956b544cebd1f5dfc87def5bd", "name": "admin"}, {"id": "c98267f0aa4144d8b7e89d9b40b46c0e", "name": "member"}, {"id": "eea735edc3e14222bae6703c2c672c02", "name": "reader"}], "catalog": [{"endpoints": [{"id": "7342dd7ff0ae438ba50ac669538feb4f", "interface": "
(...)
[Wed May 04 13:40:45.398554 2022] [wsgi:error] [pid 3007771:tid 140637774444288] [remote 192.168.1.70:59546] DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): controller:8774
[Wed May 04 13:40:45.726949 2022] [wsgi:error] [pid 3007771:tid 140637774444288] [remote 192.168.1.70:59546] DEBUG:urllib3.connectionpool:http://controller:8774 "GET /v2.1/os-services HTTP/1.1" 403 112
[Wed May 04 13:40:45.727379 2022] [wsgi:error] [pid 3007771:tid 140637774444288] [remote 192.168.1.70:59546] Recoverable error: Policy doesn't allow os_compute_api:os-services:list to be performed. (HTTP 403) (Request-ID: req-592af610-2daa-4318-8a2a-2f7fa89f83c6)
Logs indicate that horizon is using still a project-scoped token and
not a system-scoped one for these requests although ’system scope’ is
active.
Putting the same request from Openstack SDK with the same user 'admin'
results in
$ openstack compute service list
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
from cryptography.utils import int_from_bytes
+----+------------------+-------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+------------------+-------------+----------+---------+-------+----------------------------+
| 4 | nova-consoleauth | controller | nova | enabled | down | 2019-10-31T14:59:33.000000 |
| 5 | nova-scheduler | controller | nova | enabled | up | 2022-05-04T08:52:48.000000 |
| 6 | nova-conductor | controller | nova | enabled | up | 2022-05-04T08:52:42.000000 |
| 12 | nova-compute | compute3 | Crandale | enabled | up | 2022-05-04T08:52:40.000000 |
| 13 | nova-conductor | controller3 | nova | enabled | down | 2020-06-28T14:45:31.000000 |
| 14 | nova-scheduler | controller3 | nova | enabled | down | 2020-06-28T14:45:24.000000 |
+----+------------------+-------------+----------+---------+-------+—————————————--------------—+
Which indicates that role assignments to user admin are correct. The
same command with -—debug also proves that a system scoped token is
generated.
clouds.yaml for this user look like:
clouds:
mycloud-admin:
region_name: RegionOne
auth:
# domain_name: 'Default'
username: 'admin'
password: 'secretsecret'
# project_name: 'admin'
auth_url: 'https://controller:5000/v3'
#project_domain_name: 'default'
user_domain_name: 'default'
system_scope: all
interface: 'public'
identity_api_version: 3
volume_api_version: 3.68
compute_api_version: 2.90
observation is, that a system scoped token is ONLY generated if
neither 'domain_name' nor 'project_(domain_)name' are DEFINED in
clouds.yaml - only then, the service list request suceeds; otherwise a
domain resp. project scoped token is generated. Therefore they have
been commented out in clouds.yaml - in the old environment variable
logic, OS_PROJECT_ID etc. must left UNSET.
Here, another issue comes in: if we use the downloaded open-rc.sh file
from horizon for the user 'admin',
#!/usr/bin/env bash
# To use an OpenStack cloud you need to authenticate against the Identity
# service named keystone, which returns a **Token** and **Service Catalog**.
# The catalog contains the endpoints for all services the user/tenant has
# access to - such as Compute, Image Service, Identity, Object Storage, Block
# Storage, and Networking (code-named nova, glance, keystone, swift,
# cinder, and neutron).
#
# *NOTE*: Using the 3 *Identity API* does not necessarily mean any other
# OpenStack API is version 3. For example, your cloud provider may implement
# Image API v1.1, Block Storage API v2, and Compute API v2.0. OS_AUTH_URL is
# only for the Identity API served through keystone.
export OS_AUTH_URL=https://controller:5000
# With the addition of Keystone we have standardized on the term **project**
# as the entity that owns the resources.
export OS_PROJECT_ID=None
export OS_PROJECT_NAME="None"
export OS_USER_DOMAIN_NAME="Default"
if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi
export OS_PROJECT_DOMAIN_ID="None"
if [ -z "$OS_PROJECT_DOMAIN_ID" ]; then unset OS_PROJECT_DOMAIN_ID; fi
# unset v2.0 items in case set
unset OS_TENANT_ID
unset OS_TENANT_NAME
# In addition to the owning entity (tenant), OpenStack stores the entity
# performing the action as the **user**.
export OS_USERNAME="admin"
# With Keystone you pass the keystone password.
echo "Please enter your OpenStack Password for project $OS_PROJECT_NAME as user $OS_USERNAME: "
read -sr OS_PASSWORD_INPUT
export OS_PASSWORD=$OS_PASSWORD_INPUT
# If your configuration has multiple regions, we set that information here.
# OS_REGION_NAME is optional and only valid in certain environments.
export OS_REGION_NAME="RegionOne"
# Don't leave a blank variable, unset it if it was empty
if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
it is obvious, that Project_ID and project_name are defined and set to
'None'. When trying this user config for any nova-api call with the
policy set to 'rule:context_is_admin' with a system scoped call via
Openstack SDK, it throws an error 400 resp. 403 as a project scoped
token is generated resp. project 'None' results in an error; which
consequently manifests in the nova-api (here example os-quota-sets)
log as
2022-05-04 18:25:34.253 2662915 INFO nova.api.openstack.wsgi [req-6e6ee519-36f2-4937-a674-6493d8232ff6 a837078533b64bac8be7f9fb0c57ead6 - - default default] HTTP exception thrown: Project ID None is not a valid project.
2022-05-04 18:25:34.254 2662915 INFO nova.osapi_compute.wsgi.server [req-6e6ee519-36f2-4937-a674-6493d8232ff6 a837078533b64bac8be7f9fb0c57ead6 - - default default] 10.0.88.11 "GET /v2.1/os-quota-sets/None/defaults HTTP/1.1" status: 400 len: 505 time: 6.8056376
Not sure whether this is a second issue but at least it looks
inconsistent to the required behavior and would explain why the
request in horizon fails.
To me meanwhile it looks like a rather structural challenge and not
like a simple code bug ...
br c
To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1971592/+subscriptions
References