yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1547564] [NEW] nova baremetal-node-list unexpected API error 500

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: mark lewis <q@xxxxxxxxx>
Date: Fri, 19 Feb 2016 15:46:14 -0000
Reply-to: Bug 1547564 <1547564@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

$ nova baremetal-node-list

ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'ironicclient.exc.AmbiguousAuthSystem'> (HTTP 500) (Request-ID: req-78138575-cdcc-408e-a232-dc717bfe96d9)

Nova API log.
2016-02-19 15:38:04.507 DEBUG nova.osapi_compute.wsgi.server [req-48a19e8f-00dd-4fdf-a079-5cd17a030f7e None None] (31654) accepted ('10.0.0.150', 37341) from (pid=31654) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:867
2016-02-19 15:38:04.508 DEBUG keystoneauth.session [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X GET http://10.0.0.150:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}2548654be6a35b05a064f0bd15e4da50119ebeda" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}05b09de98dd6820f5766b21e415cbc27190125e6" from (pid=31654) _http_log_request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:248
2016-02-19 15:38:04.524 DEBUG keystoneauth.session [-] RESP: [200] Content-Length: 4227 X-Subject-Token: {SHA1}2548654be6a35b05a064f0bd15e4da50119ebeda Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5 Connection: Keep-Alive Date: Fri, 19 Feb 2016 15:38:04 GMT Content-Type: application/json x-openstack-request-id: req-4dd14b09-bc0e-4dfb-ae6c-76d77cbc0bd4
RESP BODY: {"token": {"methods": ["password", "token"], "roles": [{"id": "16109acaf0c640e4b96abeb2eca388e1", "name": "admin"}], "expires_at": "2016-02-19T16:38:04.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "0b130fa37b9a44a696b64559e713e032", "name": "admin"}, "catalog": "<removed>", "user": {"domain": {"id": "default", "name": "Default"}, "id": "4e6697fd84604775b097bb53bd290367", "name": "admin"}, "audit_ids": ["kEE5cM-hRHyH50QppYnuOw"], "issued_at": "2016-02-19T15:38:04.499020"}}
 from (pid=31654) _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:277
2016-02-19 15:38:04.525 INFO nova.osapi_compute.wsgi.server [req-82377dd1-15fb-439c-959b-c1afec33091b admin admin] 10.0.0.150 "GET /v2.1/0b130fa37b9a44a696b64559e713e032 HTTP/1.1" status: 404 len: 264 time: 0.0177109
2016-02-19 15:38:04.529 DEBUG nova.api.openstack.wsgi [req-24e38a23-d0ba-46aa-84ba-f6dfc27961e9 admin admin] Calling method '<bound method VersionsController.show of <nova.api.openstack.compute.versionsV21.VersionsController object at 0x7df5a10>>' from (pid=31654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:699
2016-02-19 15:38:04.530 INFO nova.osapi_compute.wsgi.server [req-24e38a23-d0ba-46aa-84ba-f6dfc27961e9 admin admin] 10.0.0.150 "GET /v2.1/ HTTP/1.1" status: 200 len: 652 time: 0.0027330
2016-02-19 15:38:04.600 DEBUG nova.api.openstack.wsgi [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] Calling method '<bound method BareMetalNodeController.index of <nova.api.openstack.compute.baremetal_nodes.BareMetalNodeController object at 0x76b68d0>>' from (pid=31654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:699
2016-02-19 15:38:04.601 ERROR nova.api.openstack.extensions [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] Unexpected exception in API method
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions Traceback (most recent call last):
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/extensions.py", line 478, in wrapped
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     return f(*args, **kwargs)
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/baremetal_nodes.py", line 105, in index
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     icli = _get_ironic_client()
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/baremetal_nodes.py", line 76, in _get_ironic_client
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     icli = ironic_client.get_client(CONF.ironic.api_version, **kwargs)
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/ironicclient/client.py", line 106, in get_client
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     raise exc.AmbiguousAuthSystem(e)
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions AmbiguousAuthSystem: Must provide Keystone credentials or user-defined endpoint and token
2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions
2016-02-19 15:38:04.601 INFO nova.api.openstack.wsgi [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'ironicclient.exc.AmbiguousAuthSystem'>
2016-02-19 15:38:04.601 DEBUG nova.api.openstack.wsgi [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'ironicclient.exc.AmbiguousAuthSystem'> from (pid=31654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1070
2016-02-19 15:38:04.602 INFO nova.osapi_compute.wsgi.server [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] 10.0.0.150 "GET /v2.1/0b130fa37b9a44a696b64559e713e032/os-baremetal-nodes HTTP/1.1" status: 500 len: 513 time: 0.0036349


* bug guideline stuff
> git log -1
commit 4bcf6529d5096c16fd1b9c418d5c40d2bb8a2831
Merge: 772e914 edc11c2
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date:   Thu Feb 18 06:50:18 2016 +0000

    Merge "Keystone support deploy in uwsgi"


> CentOS Linux release 7.2.1511 (Core) fresh install minimal
> yum -y install wget telnet git
> git clone https://git.openstack.org/openstack-dev/devstack
> and have pretty much done nothing else but read the manuals.  

Trying to launch instance in horizon and got:
Please try again later [Error: No valid host was found. There are not enough hosts available.].

Googled error and got https://ask.openstack.org/en/question/6445/no-valid-host-was-found-when-launching-an-instance/
read that, and so thought to try the bare-metal-node list and it said to report it.

Expected result: VM instance starts on default cirros image

Sorry too inexperienced with devstakc to produce script.

Good luck

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1547564

Title:
  nova baremetal-node-list unexpected API error 500

Status in OpenStack Compute (nova):
  New

Bug description:
  $ nova baremetal-node-list

  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'ironicclient.exc.AmbiguousAuthSystem'> (HTTP 500) (Request-ID: req-78138575-cdcc-408e-a232-dc717bfe96d9)

  Nova API log.
  2016-02-19 15:38:04.507 DEBUG nova.osapi_compute.wsgi.server [req-48a19e8f-00dd-4fdf-a079-5cd17a030f7e None None] (31654) accepted ('10.0.0.150', 37341) from (pid=31654) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:867
  2016-02-19 15:38:04.508 DEBUG keystoneauth.session [-] REQ: curl -g -i --cacert "/opt/stack/data/ca-bundle.pem" -X GET http://10.0.0.150:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}2548654be6a35b05a064f0bd15e4da50119ebeda" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}05b09de98dd6820f5766b21e415cbc27190125e6" from (pid=31654) _http_log_request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:248
  2016-02-19 15:38:04.524 DEBUG keystoneauth.session [-] RESP: [200] Content-Length: 4227 X-Subject-Token: {SHA1}2548654be6a35b05a064f0bd15e4da50119ebeda Vary: X-Auth-Token Keep-Alive: timeout=5, max=100 Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5 Connection: Keep-Alive Date: Fri, 19 Feb 2016 15:38:04 GMT Content-Type: application/json x-openstack-request-id: req-4dd14b09-bc0e-4dfb-ae6c-76d77cbc0bd4
  RESP BODY: {"token": {"methods": ["password", "token"], "roles": [{"id": "16109acaf0c640e4b96abeb2eca388e1", "name": "admin"}], "expires_at": "2016-02-19T16:38:04.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "0b130fa37b9a44a696b64559e713e032", "name": "admin"}, "catalog": "<removed>", "user": {"domain": {"id": "default", "name": "Default"}, "id": "4e6697fd84604775b097bb53bd290367", "name": "admin"}, "audit_ids": ["kEE5cM-hRHyH50QppYnuOw"], "issued_at": "2016-02-19T15:38:04.499020"}}
   from (pid=31654) _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:277
  2016-02-19 15:38:04.525 INFO nova.osapi_compute.wsgi.server [req-82377dd1-15fb-439c-959b-c1afec33091b admin admin] 10.0.0.150 "GET /v2.1/0b130fa37b9a44a696b64559e713e032 HTTP/1.1" status: 404 len: 264 time: 0.0177109
  2016-02-19 15:38:04.529 DEBUG nova.api.openstack.wsgi [req-24e38a23-d0ba-46aa-84ba-f6dfc27961e9 admin admin] Calling method '<bound method VersionsController.show of <nova.api.openstack.compute.versionsV21.VersionsController object at 0x7df5a10>>' from (pid=31654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:699
  2016-02-19 15:38:04.530 INFO nova.osapi_compute.wsgi.server [req-24e38a23-d0ba-46aa-84ba-f6dfc27961e9 admin admin] 10.0.0.150 "GET /v2.1/ HTTP/1.1" status: 200 len: 652 time: 0.0027330
  2016-02-19 15:38:04.600 DEBUG nova.api.openstack.wsgi [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] Calling method '<bound method BareMetalNodeController.index of <nova.api.openstack.compute.baremetal_nodes.BareMetalNodeController object at 0x76b68d0>>' from (pid=31654) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:699
  2016-02-19 15:38:04.601 ERROR nova.api.openstack.extensions [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] Unexpected exception in API method
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions Traceback (most recent call last):
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/extensions.py", line 478, in wrapped
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     return f(*args, **kwargs)
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/baremetal_nodes.py", line 105, in index
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     icli = _get_ironic_client()
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/opt/stack/nova/nova/api/openstack/compute/baremetal_nodes.py", line 76, in _get_ironic_client
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     icli = ironic_client.get_client(CONF.ironic.api_version, **kwargs)
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/ironicclient/client.py", line 106, in get_client
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions     raise exc.AmbiguousAuthSystem(e)
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions AmbiguousAuthSystem: Must provide Keystone credentials or user-defined endpoint and token
  2016-02-19 15:38:04.601 TRACE nova.api.openstack.extensions
  2016-02-19 15:38:04.601 INFO nova.api.openstack.wsgi [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'ironicclient.exc.AmbiguousAuthSystem'>
  2016-02-19 15:38:04.601 DEBUG nova.api.openstack.wsgi [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'ironicclient.exc.AmbiguousAuthSystem'> from (pid=31654) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:1070
  2016-02-19 15:38:04.602 INFO nova.osapi_compute.wsgi.server [req-78138575-cdcc-408e-a232-dc717bfe96d9 admin admin] 10.0.0.150 "GET /v2.1/0b130fa37b9a44a696b64559e713e032/os-baremetal-nodes HTTP/1.1" status: 500 len: 513 time: 0.0036349

  
  * bug guideline stuff
  > git log -1
  commit 4bcf6529d5096c16fd1b9c418d5c40d2bb8a2831
  Merge: 772e914 edc11c2
  Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
  Date:   Thu Feb 18 06:50:18 2016 +0000

      Merge "Keystone support deploy in uwsgi"

  
  > CentOS Linux release 7.2.1511 (Core) fresh install minimal
  > yum -y install wget telnet git
  > git clone https://git.openstack.org/openstack-dev/devstack
  > and have pretty much done nothing else but read the manuals.  

  Trying to launch instance in horizon and got:
  Please try again later [Error: No valid host was found. There are not enough hosts available.].

  Googled error and got https://ask.openstack.org/en/question/6445/no-valid-host-was-found-when-launching-an-instance/
  read that, and so thought to try the bare-metal-node list and it said to report it.

  Expected result: VM instance starts on default cirros image

  Sorry too inexperienced with devstakc to produce script.

  Good luck

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1547564/+subscriptions
Follow ups

[Bug 1547564] Re: nova baremetal-node-list unexpected API error 500
From: Matt Riedemann, 2016-04-18