yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1866461] [NEW] Openstack-dashboard cannot load instance console options and log options

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: ktpktro <1866461@xxxxxxxxxxxxxxxxxx>
Date: Sat, 07 Mar 2020 14:11:03 -0000
Reply-to: Bug 1866461 <1866461@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

1.Fault description
I installed the openstack-queens version, the created instance runs successfully, and I can connect through the vnc client, and the instance can access the external network. But the instance console options and log options cannot be turned on
The conductor and nova-novncproxy start normally, and there are no errors in the log

2. Failure phenomenon

The openstack-dashboard page prompts an error:
Could not load console. Please reload the page and try again
Unable to get logs for instance xxx

3.Related logs

1）Viewing the nova-api log has the following error

RROR nova.api.openstack.wsgi [req-e4268356-8f15-4c1b-b8ea-7370381e199c 245d2d1f857f45a8a337ef30129e911a b29ffeb6d6624aaaa3e2a3d39f5015ce-default default] Unexpected exception in API method: Messdaging timeout afd7fdfdfdfd
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return f (* args, ** kwargs)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 108, in wrapper
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return func (* args, ** kwargs)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/remote_consoles.py", line 140, in get_serial_console
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi console_type)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 196, in wrapped
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return function (self, context, instance, * args, ** kwargs)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 186, in inner
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return f (self, context, instance, * args, ** kw)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3798, in get_serial_console
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi instance = instance, console_type = console_type)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 696, in get_serial_console
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi instance = instance, console_type = console_type)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 181, in call
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi transport_options = self.transport_options)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 129, in _send
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi transport_options = transport_options)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 54, in send
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi transport_options = transport_options)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 634, in _send
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi call_monitor_timeout)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 523, in wait
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi message = self.waiters.get (msg_id, timeout = timeout)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 401, in get
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi 'to message ID% s'% msg_id)
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi MessagingTimeout: Timed out waiting for a reply to message ID 187bbf7579cf40d8ae5d68efd3b33c09
2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi
2020-03-07 21: 53: 11.399 102531 INFO nova.api.openstack.wsgi [req-e4268356-8f15-4c1b-b8ea-7370381e199c 245d2d1f857f45a8a337ef30129e911a b29ffeb6d6624 aaa3e2a3d39f5015ce-http default exception exception at HTTP default exception exception : //bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'>
2020-03-07 21: 53: 11.412 102531 INFO nova.osapi_compute.wsgi.server [req-e4268356-8f15-4c1b-b8ea-7370381e199c 245d2d1f857f45a8a337ef30129e911a b29ffeb6d6624aaaa3e2a3d39f5015v2.12.15 default 5.0 -cb82-46c7-a944-83a16e409f2a / action HTTP / 1.1 "status: 500 len: 649 time: 60.0952358

2）Viewing the metadata-agent log has the following error

= ERROR REPORT ==== 4-Mar-2020 :: 17: 48: 11 ===
Channel error on connection <0.1639.0> (192.168.10.18.252:33646-> 192.168.18.250:5672, vhost: '/', user: 'openstack'), channel 1:
operation basic.publish caused a channel exception not_found: no exchange 'reply_be15918dbbf649bfb90793e0fac59136' in vhost '/'

3）Get instance vnc link error (instance ID or instance name)
[root @ controller1 ~ (keystone_admin)] $ nova get-vnc-console b5ee9b49-cae0-49ec-8f66-67e0324ffcf5 novnc
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-e5ba92b3-3bdf-4244-b2bf-feef1514bf9a)

add --debug option

[root@controller1 ~(keystone_admin)]$ nova --debug get-vnc-console test novnc
DEBUG (extension:189) found extension EntryPoint.parse('v1password = swiftclient.authv1:PasswordLoader')
DEBUG (extension:189) found extension EntryPoint.parse('noauth = cinderclient.contrib.noauth:CinderNoAuthLoader')
DEBUG (extension:189) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
DEBUG (extension:189) found extension EntryPoint.parse('none = keystoneauth1.loading._plugins.noauth:NoAuth')
DEBUG (extension:189) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1')
DEBUG (extension:189) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
DEBUG (extension:189) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
DEBUG (extension:189) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos')
DEBUG (extension:189) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP')
DEBUG (extension:189) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials')
DEBUG (extension:189) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth')
DEBUG (extension:189) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
DEBUG (extension:189) found extension EntryPoint.parse('v3multifactor = keystoneauth1.loading._plugins.identity.v3:MultiFactor')
DEBUG (extension:189) found extension EntryPoint.parse('v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential')
DEBUG (extension:189) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos')
DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:5000/v3 -H "Accept: application/json" -H "User-Agent: nova keystoneauth1/3.17.1 python-requests/2.21.0 CPython/2.7.5"
DEBUG (connectionpool:205) Starting new HTTP connection (1): controller1:5000
DEBUG (connectionpool:393) http://controller1:5000 "GET /v3 HTTP/1.1" 200 251
DEBUG (session:525) RESP: [200] Connection: Keep-Alive Content-Length: 251 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:32 GMT Keep-Alive: timeout=5, max=100 Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 Vary: X-Auth-Token x-openstack-request-id: req-6c150b61-6228-46be-8a35-c2dd06e14f6f
DEBUG (session:557) RESP BODY: {"version": {"status": "stable", "updated": "2019-07-19T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.13", "links": [{"href": "http://controller1:5000/v3/";, "rel": "self"}]}}
DEBUG (session:927) GET call to http://controller1:5000/v3 used request id req-6c150b61-6228-46be-8a35-c2dd06e14f6f
DEBUG (base:182) Making authentication request to http://controller1:5000/v3/auth/tokens
DEBUG (connectionpool:393) http://controller1:5000 "POST /v3/auth/tokens HTTP/1.1" 201 3419
DEBUG (base:187) {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "33568cccb0c04ed39df1f98edb297b98", "name": "member"}, {"id": "b72b3e57b36f4f02ab0ce0af7ffc23e1", "name": "reader"}, {"id": "03a4802c9b4344008865934054898b5d", "name": "admin"}], "expires_at": "2020-03-07T15:08:32.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "b29ffeb6d6624aaaa3e2a3d39f5015ce", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller1:8778";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "4a1b77e7b9124f4e953f15df6010b8c3"}, {"url": "http://controller1:8778";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "9a4d473f9146485c93e94b6130525fe5"}, {"url": "http://controller1:8778";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "ee594ad9f8e4435e8716cf872592ed26"}], "type": "placement", "id": "03360ee1973e4902bdf7d52bfe30da1e", "name": "placement"}, {"endpoints": [{"url": "http://controller1:9292";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8086370825a7478b996cf0beb6630a13"}, {"url": "http://controller1:9292";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "88fc132ba819455397da7d42d65347a3"}, {"url": "http://controller1:9292";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b56f23a726fb4d10af98382f1b31e9fd"}], "type": "image", "id": "49c5e33bb4c048f9b3e8da368b5ff101", "name": "glance"}, {"endpoints": [{"url": "http://compute1:9696";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "2d2cec7c254442689d38255277ff7f7b"}, {"url": "http://compute1:9696";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b4b3725328a844b0bc0dc4f41f20b711"}, {"url": "http://compute1:9696";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "bddcf69815b74ea88d1a42c9533af8a2"}], "type": "network", "id": "595424677f75472abf22d7346975ca4e", "name": "neutron"}, {"endpoints": [{"url": "http://controller1:8774/v2.1";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "09f804ac83464929806e2a7b305e743a"}, {"url": "http://controller1:8774/v2.1";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "23df3d298c3b4217a3b80b35831e004f"}, {"url": "http://controller1:8774/v2.1";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "9df86c4c7aa9425288c948c1c79f1fd1"}], "type": "compute", "id": "b70e94dab44745bf9043ae82880f8b97", "name": "nova"}, {"endpoints": [{"url": "http://controller1:5000/v3/";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1a48327c824842a7bc39eddafd2efa36"}, {"url": "http://controller1:5000/v3/";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "42030d5c281a46a395dcaa0658b21e49"}, {"url": "http://controller1:5000/v3/";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c6c2ec4063b44550b845ed25a2465b19"}], "type": "identity", "id": "c7e71c5d1e9849c094c9438cec0d4378", "name": "keystone"}], "user": {"domain": {"id": "default", "name": "Default"}, "password_expires_at": null, "name": "admin", "id": "245d2d1f857f45a8a337ef30129e911a"}, "audit_ids": ["7Q6hW_XAQQSos_3JATdcjA"], "issued_at": "2020-03-07T14:08:32.000000Z"}}
REQ: curl -g -i -X GET http://controller1:8774/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1"
DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:8774/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1"
DEBUG (connectionpool:205) Starting new HTTP connection (1): controller1:8774
DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1 HTTP/1.1" 302 0
RESP: [302] Connection: keep-alive Content-Length: 0 Content-Type: text/plain; charset=utf8 Date: Sat, 07 Mar 2020 14:08:33 GMT Location: http://controller1:8774/v2.1/ X-Compute-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480 X-Openstack-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480
DEBUG (session:525) RESP: [302] Connection: keep-alive Content-Length: 0 Content-Type: text/plain; charset=utf8 Date: Sat, 07 Mar 2020 14:08:33 GMT Location: http://controller1:8774/v2.1/ X-Compute-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480 X-Openstack-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480
RESP BODY: Omitted, Content-Type is set to text/plain; charset=utf8. Only application/json responses have their bodies logged.
DEBUG (session:557) RESP BODY: Omitted, Content-Type is set to text/plain; charset=utf8. Only application/json responses have their bodies logged.
DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1/ HTTP/1.1" 200 385
RESP: [200] Connection: keep-alive Content-Length: 385 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:33 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646 X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646
DEBUG (session:525) RESP: [200] Connection: keep-alive Content-Length: 385 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:33 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646 X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646
RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://controller1:8774/v2.1/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.60", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}}
DEBUG (session:557) RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://controller1:8774/v2.1/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.60", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}}
GET call to compute for http://controller1:8774/v2.1/ used request id req-d02f0acb-2396-4b45-aef2-0e1a7d171646
DEBUG (session:918) GET call to compute for http://controller1:8774/v2.1/ used request id req-d02f0acb-2396-4b45-aef2-0e1a7d171646
DEBUG (extension:189) found extension EntryPoint.parse('v1password = swiftclient.authv1:PasswordLoader')
DEBUG (extension:189) found extension EntryPoint.parse('noauth = cinderclient.contrib.noauth:CinderNoAuthLoader')
DEBUG (extension:189) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
DEBUG (extension:189) found extension EntryPoint.parse('none = keystoneauth1.loading._plugins.noauth:NoAuth')
DEBUG (extension:189) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1')
DEBUG (extension:189) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
DEBUG (extension:189) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
DEBUG (extension:189) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos')
DEBUG (extension:189) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP')
DEBUG (extension:189) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
DEBUG (extension:189) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials')
DEBUG (extension:189) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth')
DEBUG (extension:189) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
DEBUG (extension:189) found extension EntryPoint.parse('v3multifactor = keystoneauth1.loading._plugins.identity.v3:MultiFactor')
DEBUG (extension:189) found extension EntryPoint.parse('v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential')
DEBUG (extension:189) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
DEBUG (extension:189) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos')
DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:8774/v2.1/servers?name=test -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.60" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1" -H "X-OpenStack-Nova-API-Version: 2.60"
DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1/servers?name=test HTTP/1.1" 200 291
DEBUG (session:525) RESP: [200] Connection: keep-alive Content-Length: 291 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:33 GMT Openstack-Api-Version: compute 2.60 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-25a905a2-ae42-459d-a14d-ce53e40d413a X-Openstack-Nova-Api-Version: 2.60 X-Openstack-Request-Id: req-25a905a2-ae42-459d-a14d-ce53e40d413a
DEBUG (session:557) RESP BODY: {"servers": [{"id": "5ddbb658-cb82-46c7-a944-83a16e409f2a", "links": [{"href": "http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "self"}, {"href": "http://controller1:8774/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "bookmark"}], "name": "test"}]}
DEBUG (session:918) GET call to compute for http://controller1:8774/v2.1/servers?name=test used request id req-25a905a2-ae42-459d-a14d-ce53e40d413a
DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.60" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1" -H "X-OpenStack-Nova-API-Version: 2.60"
DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a HTTP/1.1" 200 1895
DEBUG (session:525) RESP: [200] Connection: keep-alive Content-Length: 1895 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:34 GMT Openstack-Api-Version: compute 2.60 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-d6461ab1-4be3-446c-bcf4-ad0c2a458e24 X-Openstack-Nova-Api-Version: 2.60 X-Openstack-Request-Id: req-d6461ab1-4be3-446c-bcf4-ad0c2a458e24
DEBUG (session:557) RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"own": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:36:51:a0", "version": 4, "addr": "172.18.50.24", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "self"}, {"href": "http://controller1:8774/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "bookmark"}], "image": {"id": "f9d943ca-8169-4cbb-98bf-1bbfa2e01a85", "links": [{"href": "http://controller1:8774/images/f9d943ca-8169-4cbb-98bf-1bbfa2e01a85";, "rel": "bookmark"}]}, "OS-EXT-SRV-ATTR:user_data": null, "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000004", "OS-EXT-SRV-ATTR:root_device_name": "/dev/vda", "OS-SRV-USG:launched_at": "2020-03-07T04:03:43.000000", "flavor": {"ephemeral": 0, "ram": 1024, "original_name": "test", "vcpus": 1, "extra_specs": {}, "swap": 0, "disk": 20}, "id": "5ddbb658-cb82-46c7-a944-83a16e409f2a", "security_groups": [{"name": "default"}], "OS-SRV-USG:terminated_at": null, "os-extended-volumes:volumes_attached": [], "user_id": "245d2d1f857f45a8a337ef30129e911a", "OS-EXT-SRV-ATTR:hostname": "test", "OS-DCF:diskConfig": "AUTO", "accessIPv4": "", "accessIPv6": "", "OS-EXT-SRV-ATTR:reservation_id": "r-3qqkgc7e", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "status": "ACTIVE", "OS-EXT-SRV-ATTR:ramdisk_id": "", "updated": "2020-03-07T04:03:44Z", "hostId": "28a935781ac4dc2149c3e7e75870cf8fcfef3b5ca94dd938ab2315e6", "OS-EXT-SRV-ATTR:host": "compute1", "description": null, "tags": [], "key_name": null, "OS-EXT-SRV-ATTR:kernel_id": "", "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute1", "locked": false, "name": "test", "OS-EXT-SRV-ATTR:launch_index": 0, "created": "2020-03-07T04:03:26Z", "tenant_id": "b29ffeb6d6624aaaa3e2a3d39f5015ce", "host_status": "UP", "metadata": {}}}
DEBUG (session:918) GET call to compute for http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a used request id req-d6461ab1-4be3-446c-bcf4-ad0c2a458e24
DEBUG (session:494) REQ: curl -g -i -X POST http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a/remote-consoles -H "Accept: application/json" -H "Content-Type: application/json" -H "OpenStack-API-Version: compute 2.60" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1" -H "X-OpenStack-Nova-API-Version: 2.60" -d '{"remote_console": {"type": "novnc", "protocol": "vnc"}}'
DEBUG (connectionpool:393) http://controller1:8774 "POST /v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a/remote-consoles HTTP/1.1" 500 216
DEBUG (session:525) RESP: [500] Connection: keep-alive Content-Length: 216 Content-Type: application/json; charset=UTF-8 Date: Sat, 07 Mar 2020 14:09:34 GMT Openstack-Api-Version: compute 2.60 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-9ae59b20-115c-4730-9b5f-305d5fc772e1 X-Openstack-Nova-Api-Version: 2.60 X-Openstack-Request-Id: req-9ae59b20-115c-4730-9b5f-305d5fc772e1
DEBUG (session:557) RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}
DEBUG (session:918) POST call to compute for http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a/remote-consoles used request id req-9ae59b20-115c-4730-9b5f-305d5fc772e1
DEBUG (shell:812) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ae59b20-115c-4730-9b5f-305d5fc772e1)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 810, in main
    OpenStackComputeShell().main(argv)
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 732, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 2747, in do_get_vnc_console
    data = server.get_vnc_console(args.console_type)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 108, in get_vnc_console
    return self.manager.get_vnc_console(self, console_type)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 955, in get_vnc_console
    return self.get_console_url(server, console_type)
  File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 393, in substitution
    return methods[-1].func(obj, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1043, in get_console_url
    resp, body = self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 392, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 78, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ae59b20-115c-4730-9b5f-305d5fc772e1)
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ae59b20-115c-4730-9b5f-305d5fc772e1)

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1866461

Title:
  Openstack-dashboard cannot load instance console options and log
  options

Status in OpenStack Compute (nova):
  New

Bug description:
  1.Fault description
  I installed the openstack-queens version, the created instance runs successfully, and I can connect through the vnc client, and the instance can access the external network. But the instance console options and log options cannot be turned on
  The conductor and nova-novncproxy start normally, and there are no errors in the log

  2. Failure phenomenon

  The openstack-dashboard page prompts an error:
  Could not load console. Please reload the page and try again
  Unable to get logs for instance xxx

  3.Related logs

  1）Viewing the nova-api log has the following error

  RROR nova.api.openstack.wsgi [req-e4268356-8f15-4c1b-b8ea-7370381e199c 245d2d1f857f45a8a337ef30129e911a b29ffeb6d6624aaaa3e2a3d39f5015ce-default default] Unexpected exception in API method: Messdaging timeout afd7fdfdfdfd
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return f (* args, ** kwargs)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 108, in wrapper
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return func (* args, ** kwargs)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/remote_consoles.py", line 140, in get_serial_console
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi console_type)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 196, in wrapped
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return function (self, context, instance, * args, ** kwargs)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 186, in inner
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi return f (self, context, instance, * args, ** kw)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3798, in get_serial_console
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi instance = instance, console_type = console_type)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/rpcapi.py", line 696, in get_serial_console
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi instance = instance, console_type = console_type)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 181, in call
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi transport_options = self.transport_options)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 129, in _send
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi transport_options = transport_options)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 54, in send
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi transport_options = transport_options)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 634, in _send
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi call_monitor_timeout)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 523, in wait
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi message = self.waiters.get (msg_id, timeout = timeout)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 401, in get
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi 'to message ID% s'% msg_id)
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi MessagingTimeout: Timed out waiting for a reply to message ID 187bbf7579cf40d8ae5d68efd3b33c09
  2020-03-07 21: 53: 11.391 102531 ERROR nova.api.openstack.wsgi
  2020-03-07 21: 53: 11.399 102531 INFO nova.api.openstack.wsgi [req-e4268356-8f15-4c1b-b8ea-7370381e199c 245d2d1f857f45a8a337ef30129e911a b29ffeb6d6624 aaa3e2a3d39f5015ce-http default exception exception at HTTP default exception exception : //bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'>
  2020-03-07 21: 53: 11.412 102531 INFO nova.osapi_compute.wsgi.server [req-e4268356-8f15-4c1b-b8ea-7370381e199c 245d2d1f857f45a8a337ef30129e911a b29ffeb6d6624aaaa3e2a3d39f5015v2.12.15 default 5.0 -cb82-46c7-a944-83a16e409f2a / action HTTP / 1.1 "status: 500 len: 649 time: 60.0952358

  2）Viewing the metadata-agent log has the following error

  = ERROR REPORT ==== 4-Mar-2020 :: 17: 48: 11 ===
  Channel error on connection <0.1639.0> (192.168.10.18.252:33646-> 192.168.18.250:5672, vhost: '/', user: 'openstack'), channel 1:
  operation basic.publish caused a channel exception not_found: no exchange 'reply_be15918dbbf649bfb90793e0fac59136' in vhost '/'

  3）Get instance vnc link error (instance ID or instance name)
  [root @ controller1 ~ (keystone_admin)] $ nova get-vnc-console b5ee9b49-cae0-49ec-8f66-67e0324ffcf5 novnc
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-e5ba92b3-3bdf-4244-b2bf-feef1514bf9a)

  add --debug option

  [root@controller1 ~(keystone_admin)]$ nova --debug get-vnc-console test novnc
  DEBUG (extension:189) found extension EntryPoint.parse('v1password = swiftclient.authv1:PasswordLoader')
  DEBUG (extension:189) found extension EntryPoint.parse('noauth = cinderclient.contrib.noauth:CinderNoAuthLoader')
  DEBUG (extension:189) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
  DEBUG (extension:189) found extension EntryPoint.parse('none = keystoneauth1.loading._plugins.noauth:NoAuth')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1')
  DEBUG (extension:189) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
  DEBUG (extension:189) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
  DEBUG (extension:189) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos')
  DEBUG (extension:189) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP')
  DEBUG (extension:189) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials')
  DEBUG (extension:189) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth')
  DEBUG (extension:189) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
  DEBUG (extension:189) found extension EntryPoint.parse('v3multifactor = keystoneauth1.loading._plugins.identity.v3:MultiFactor')
  DEBUG (extension:189) found extension EntryPoint.parse('v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential')
  DEBUG (extension:189) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos')
  DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:5000/v3 -H "Accept: application/json" -H "User-Agent: nova keystoneauth1/3.17.1 python-requests/2.21.0 CPython/2.7.5"
  DEBUG (connectionpool:205) Starting new HTTP connection (1): controller1:5000
  DEBUG (connectionpool:393) http://controller1:5000 "GET /v3 HTTP/1.1" 200 251
  DEBUG (session:525) RESP: [200] Connection: Keep-Alive Content-Length: 251 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:32 GMT Keep-Alive: timeout=5, max=100 Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 Vary: X-Auth-Token x-openstack-request-id: req-6c150b61-6228-46be-8a35-c2dd06e14f6f
  DEBUG (session:557) RESP BODY: {"version": {"status": "stable", "updated": "2019-07-19T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.13", "links": [{"href": "http://controller1:5000/v3/";, "rel": "self"}]}}
  DEBUG (session:927) GET call to http://controller1:5000/v3 used request id req-6c150b61-6228-46be-8a35-c2dd06e14f6f
  DEBUG (base:182) Making authentication request to http://controller1:5000/v3/auth/tokens
  DEBUG (connectionpool:393) http://controller1:5000 "POST /v3/auth/tokens HTTP/1.1" 201 3419
  DEBUG (base:187) {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "33568cccb0c04ed39df1f98edb297b98", "name": "member"}, {"id": "b72b3e57b36f4f02ab0ce0af7ffc23e1", "name": "reader"}, {"id": "03a4802c9b4344008865934054898b5d", "name": "admin"}], "expires_at": "2020-03-07T15:08:32.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "b29ffeb6d6624aaaa3e2a3d39f5015ce", "name": "admin"}, "catalog": [{"endpoints": [{"url": "http://controller1:8778";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "4a1b77e7b9124f4e953f15df6010b8c3"}, {"url": "http://controller1:8778";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "9a4d473f9146485c93e94b6130525fe5"}, {"url": "http://controller1:8778";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "ee594ad9f8e4435e8716cf872592ed26"}], "type": "placement", "id": "03360ee1973e4902bdf7d52bfe30da1e", "name": "placement"}, {"endpoints": [{"url": "http://controller1:9292";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "8086370825a7478b996cf0beb6630a13"}, {"url": "http://controller1:9292";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "88fc132ba819455397da7d42d65347a3"}, {"url": "http://controller1:9292";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "b56f23a726fb4d10af98382f1b31e9fd"}], "type": "image", "id": "49c5e33bb4c048f9b3e8da368b5ff101", "name": "glance"}, {"endpoints": [{"url": "http://compute1:9696";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "2d2cec7c254442689d38255277ff7f7b"}, {"url": "http://compute1:9696";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "b4b3725328a844b0bc0dc4f41f20b711"}, {"url": "http://compute1:9696";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "bddcf69815b74ea88d1a42c9533af8a2"}], "type": "network", "id": "595424677f75472abf22d7346975ca4e", "name": "neutron"}, {"endpoints": [{"url": "http://controller1:8774/v2.1";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "09f804ac83464929806e2a7b305e743a"}, {"url": "http://controller1:8774/v2.1";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "23df3d298c3b4217a3b80b35831e004f"}, {"url": "http://controller1:8774/v2.1";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "9df86c4c7aa9425288c948c1c79f1fd1"}], "type": "compute", "id": "b70e94dab44745bf9043ae82880f8b97", "name": "nova"}, {"endpoints": [{"url": "http://controller1:5000/v3/";, "interface": "internal", "region": "RegionOne", "region_id": "RegionOne", "id": "1a48327c824842a7bc39eddafd2efa36"}, {"url": "http://controller1:5000/v3/";, "interface": "admin", "region": "RegionOne", "region_id": "RegionOne", "id": "42030d5c281a46a395dcaa0658b21e49"}, {"url": "http://controller1:5000/v3/";, "interface": "public", "region": "RegionOne", "region_id": "RegionOne", "id": "c6c2ec4063b44550b845ed25a2465b19"}], "type": "identity", "id": "c7e71c5d1e9849c094c9438cec0d4378", "name": "keystone"}], "user": {"domain": {"id": "default", "name": "Default"}, "password_expires_at": null, "name": "admin", "id": "245d2d1f857f45a8a337ef30129e911a"}, "audit_ids": ["7Q6hW_XAQQSos_3JATdcjA"], "issued_at": "2020-03-07T14:08:32.000000Z"}}
  REQ: curl -g -i -X GET http://controller1:8774/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1"
  DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:8774/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1"
  DEBUG (connectionpool:205) Starting new HTTP connection (1): controller1:8774
  DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1 HTTP/1.1" 302 0
  RESP: [302] Connection: keep-alive Content-Length: 0 Content-Type: text/plain; charset=utf8 Date: Sat, 07 Mar 2020 14:08:33 GMT Location: http://controller1:8774/v2.1/ X-Compute-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480 X-Openstack-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480
  DEBUG (session:525) RESP: [302] Connection: keep-alive Content-Length: 0 Content-Type: text/plain; charset=utf8 Date: Sat, 07 Mar 2020 14:08:33 GMT Location: http://controller1:8774/v2.1/ X-Compute-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480 X-Openstack-Request-Id: req-f512212a-38c9-4248-9090-be31841e7480
  RESP BODY: Omitted, Content-Type is set to text/plain; charset=utf8. Only application/json responses have their bodies logged.
  DEBUG (session:557) RESP BODY: Omitted, Content-Type is set to text/plain; charset=utf8. Only application/json responses have their bodies logged.
  DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1/ HTTP/1.1" 200 385
  RESP: [200] Connection: keep-alive Content-Length: 385 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:33 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646 X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646
  DEBUG (session:525) RESP: [200] Connection: keep-alive Content-Length: 385 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:33 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646 X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-d02f0acb-2396-4b45-aef2-0e1a7d171646
  RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://controller1:8774/v2.1/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.60", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}}
  DEBUG (session:557) RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://controller1:8774/v2.1/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.60", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}}
  GET call to compute for http://controller1:8774/v2.1/ used request id req-d02f0acb-2396-4b45-aef2-0e1a7d171646
  DEBUG (session:918) GET call to compute for http://controller1:8774/v2.1/ used request id req-d02f0acb-2396-4b45-aef2-0e1a7d171646
  DEBUG (extension:189) found extension EntryPoint.parse('v1password = swiftclient.authv1:PasswordLoader')
  DEBUG (extension:189) found extension EntryPoint.parse('noauth = cinderclient.contrib.noauth:CinderNoAuthLoader')
  DEBUG (extension:189) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token')
  DEBUG (extension:189) found extension EntryPoint.parse('none = keystoneauth1.loading._plugins.noauth:NoAuth')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1')
  DEBUG (extension:189) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode')
  DEBUG (extension:189) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword')
  DEBUG (extension:189) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos')
  DEBUG (extension:189) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP')
  DEBUG (extension:189) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token')
  DEBUG (extension:189) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials')
  DEBUG (extension:189) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth')
  DEBUG (extension:189) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token')
  DEBUG (extension:189) found extension EntryPoint.parse('v3multifactor = keystoneauth1.loading._plugins.identity.v3:MultiFactor')
  DEBUG (extension:189) found extension EntryPoint.parse('v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential')
  DEBUG (extension:189) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password')
  DEBUG (extension:189) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos')
  DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:8774/v2.1/servers?name=test -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.60" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1" -H "X-OpenStack-Nova-API-Version: 2.60"
  DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1/servers?name=test HTTP/1.1" 200 291
  DEBUG (session:525) RESP: [200] Connection: keep-alive Content-Length: 291 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:33 GMT Openstack-Api-Version: compute 2.60 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-25a905a2-ae42-459d-a14d-ce53e40d413a X-Openstack-Nova-Api-Version: 2.60 X-Openstack-Request-Id: req-25a905a2-ae42-459d-a14d-ce53e40d413a
  DEBUG (session:557) RESP BODY: {"servers": [{"id": "5ddbb658-cb82-46c7-a944-83a16e409f2a", "links": [{"href": "http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "self"}, {"href": "http://controller1:8774/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "bookmark"}], "name": "test"}]}
  DEBUG (session:918) GET call to compute for http://controller1:8774/v2.1/servers?name=test used request id req-25a905a2-ae42-459d-a14d-ce53e40d413a
  DEBUG (session:494) REQ: curl -g -i -X GET http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.60" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1" -H "X-OpenStack-Nova-API-Version: 2.60"
  DEBUG (connectionpool:393) http://controller1:8774 "GET /v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a HTTP/1.1" 200 1895
  DEBUG (session:525) RESP: [200] Connection: keep-alive Content-Length: 1895 Content-Type: application/json Date: Sat, 07 Mar 2020 14:08:34 GMT Openstack-Api-Version: compute 2.60 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-d6461ab1-4be3-446c-bcf4-ad0c2a458e24 X-Openstack-Nova-Api-Version: 2.60 X-Openstack-Request-Id: req-d6461ab1-4be3-446c-bcf4-ad0c2a458e24
  DEBUG (session:557) RESP BODY: {"server": {"OS-EXT-STS:task_state": null, "addresses": {"own": [{"OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:36:51:a0", "version": 4, "addr": "172.18.50.24", "OS-EXT-IPS:type": "fixed"}]}, "links": [{"href": "http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "self"}, {"href": "http://controller1:8774/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a";, "rel": "bookmark"}], "image": {"id": "f9d943ca-8169-4cbb-98bf-1bbfa2e01a85", "links": [{"href": "http://controller1:8774/images/f9d943ca-8169-4cbb-98bf-1bbfa2e01a85";, "rel": "bookmark"}]}, "OS-EXT-SRV-ATTR:user_data": null, "OS-EXT-STS:vm_state": "active", "OS-EXT-SRV-ATTR:instance_name": "instance-00000004", "OS-EXT-SRV-ATTR:root_device_name": "/dev/vda", "OS-SRV-USG:launched_at": "2020-03-07T04:03:43.000000", "flavor": {"ephemeral": 0, "ram": 1024, "original_name": "test", "vcpus": 1, "extra_specs": {}, "swap": 0, "disk": 20}, "id": "5ddbb658-cb82-46c7-a944-83a16e409f2a", "security_groups": [{"name": "default"}], "OS-SRV-USG:terminated_at": null, "os-extended-volumes:volumes_attached": [], "user_id": "245d2d1f857f45a8a337ef30129e911a", "OS-EXT-SRV-ATTR:hostname": "test", "OS-DCF:diskConfig": "AUTO", "accessIPv4": "", "accessIPv6": "", "OS-EXT-SRV-ATTR:reservation_id": "r-3qqkgc7e", "progress": 0, "OS-EXT-STS:power_state": 1, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "status": "ACTIVE", "OS-EXT-SRV-ATTR:ramdisk_id": "", "updated": "2020-03-07T04:03:44Z", "hostId": "28a935781ac4dc2149c3e7e75870cf8fcfef3b5ca94dd938ab2315e6", "OS-EXT-SRV-ATTR:host": "compute1", "description": null, "tags": [], "key_name": null, "OS-EXT-SRV-ATTR:kernel_id": "", "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute1", "locked": false, "name": "test", "OS-EXT-SRV-ATTR:launch_index": 0, "created": "2020-03-07T04:03:26Z", "tenant_id": "b29ffeb6d6624aaaa3e2a3d39f5015ce", "host_status": "UP", "metadata": {}}}
  DEBUG (session:918) GET call to compute for http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a used request id req-d6461ab1-4be3-446c-bcf4-ad0c2a458e24
  DEBUG (session:494) REQ: curl -g -i -X POST http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a/remote-consoles -H "Accept: application/json" -H "Content-Type: application/json" -H "OpenStack-API-Version: compute 2.60" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}04450738ac6a65042bb5a92fb5900b85f7235bb097e35291767b731544aae9a1" -H "X-OpenStack-Nova-API-Version: 2.60" -d '{"remote_console": {"type": "novnc", "protocol": "vnc"}}'
  DEBUG (connectionpool:393) http://controller1:8774 "POST /v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a/remote-consoles HTTP/1.1" 500 216
  DEBUG (session:525) RESP: [500] Connection: keep-alive Content-Length: 216 Content-Type: application/json; charset=UTF-8 Date: Sat, 07 Mar 2020 14:09:34 GMT Openstack-Api-Version: compute 2.60 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-9ae59b20-115c-4730-9b5f-305d5fc772e1 X-Openstack-Nova-Api-Version: 2.60 X-Openstack-Request-Id: req-9ae59b20-115c-4730-9b5f-305d5fc772e1
  DEBUG (session:557) RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}
  DEBUG (session:918) POST call to compute for http://controller1:8774/v2.1/servers/5ddbb658-cb82-46c7-a944-83a16e409f2a/remote-consoles used request id req-9ae59b20-115c-4730-9b5f-305d5fc772e1
  DEBUG (shell:812) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ae59b20-115c-4730-9b5f-305d5fc772e1)
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 810, in main
      OpenStackComputeShell().main(argv)
    File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 732, in main
      args.func(self.cs, args)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 2747, in do_get_vnc_console
      data = server.get_vnc_console(args.console_type)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 108, in get_vnc_console
      return self.manager.get_vnc_console(self, console_type)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 955, in get_vnc_console
      return self.get_console_url(server, console_type)
    File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 393, in substitution
      return methods[-1].func(obj, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1043, in get_console_url
      resp, body = self.api.client.post(url, body=body)
    File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 392, in post
      return self.request(url, 'POST', **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 78, in request
      raise exceptions.from_response(resp, body, url, method)
  ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ae59b20-115c-4730-9b5f-305d5fc772e1)
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ae59b20-115c-4730-9b5f-305d5fc772e1)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1866461/+subscriptions
Follow ups

[Bug 1866461] Re: Openstack-dashboard cannot load instance console options and log options
From: Launchpad Bug Tracker, 2020-06-15