yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1659366] Re: unexpected API error when executing nova boot

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem@xxxxxxxxxx>
Date: Sun, 05 Feb 2017 20:33:41 -0000
Reply-to: Bug 1659366 <1659366@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
This needs the n-api and n-cpu logs to see what's failing, or at least
the relevant trace from those logs with the error.

** Changed in: nova
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1659366

Title:
  unexpected API error when executing nova boot

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Description
  ===========================================

  When i launched a instance by executing the command:

    $ nova boot --flavor m1.tiny --image cirros \
      --nic net-id=b4770dbb-b1aa-47a8-ac8c-20bbf528b764 \
      --security-group default --key-name mykey private-instance

  An unexpected API Error came out

    class 'oslo_messaging.exceptions.MessagingTimeout'

  Steps to reproduce
  ===========================================

  I followed the install guides on http://docs.openstack.org/liberty
  /install-guide-rdo/common/conventions.html. And before launching a
  instance, everything seems fine. The openstack client command is
  below:

  [root@controller1 /]# nova --debug boot --flavor m1.tiny --image cirros --nic net-id=b4770dbb-b1aa-47a8-ac8c-20bbf528b764 --security-group default --key-name mykey private-instance
  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:5000/v3 -H "Accept: application/json" -H "User-Agent: python-keystoneclient"
  INFO (connectionpool:213) Starting new HTTP connection (1): controller1
  DEBUG (connectionpool:395) "GET /v3 HTTP/1.1" 200 250
  DEBUG (session:215) RESP: [200] Date: Wed, 25 Jan 2017 17:10:50 GMT Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 Vary: X-Auth-Token x-openstack-request-id: req-90ba7bc5-4d52-43ce-9c0c-feefa3c79a58 Content-Length: 250 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: application/json
  RESP BODY: {"version": {"status": "stable", "updated": "2015-03-30T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": [{"href": "http://controller1:5000/v3/";, "rel": "self"}]}}

  DEBUG (base:188) Making authentication request to http://controller1:5000/v3/auth/tokens
  DEBUG (connectionpool:395) "POST /v3/auth/tokens HTTP/1.1" 201 2811
  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:8774/v2/ -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281"
  INFO (connectionpool:213) Starting new HTTP connection (1): controller1
  DEBUG (connectionpool:395) "GET /v2/ HTTP/1.1" 200 376
  DEBUG (session:215) RESP: [200] Content-Type: application/json Content-Length: 376 X-Compute-Request-Id: req-4a34ea1c-9a89-4175-b0b3-7b246374edb0 Date: Wed, 25 Jan 2017 17:10:51 GMT Connection: keep-alive
  RESP BODY: {"version": {"status": "SUPPORTED", "updated": "2011-01-21T11:33:21Z", "links": [{"href": "http://controller1:8774/v2/";, "rel": "self"}, {"href": "http://docs.openstack.org/";, "type": "text/html", "rel": "describedby"}], "min_version": "", "version": "", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2"}], "id": "v2.0"}}

  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/images -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281"
  DEBUG (connectionpool:395) "GET /v2/6dfd36b312394e358df805889fb136ee/images HTTP/1.1" 200 498
  DEBUG (session:215) RESP: [200] Content-Type: application/json Content-Length: 498 X-Compute-Request-Id: req-8e720272-ec12-43a1-a98c-65207a83266c Date: Wed, 25 Jan 2017 17:10:52 GMT Connection: keep-alive
  RESP BODY: {"images": [{"id": "dc323229-2338-41cc-8635-cd701836cf50", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/images/dc323229-2338-41cc-8635-cd701836cf50";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/images/dc323229-2338-41cc-8635-cd701836cf50";, "rel": "bookmark"}, {"href": "http://controller1:9292/images/dc323229-2338-41cc-8635-cd701836cf50";, "type": "application/vnd.openstack.image", "rel": "alternate"}], "name": "cirros"}]}

  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/images/dc323229-2338-41cc-8635-cd701836cf50 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281"
  DEBUG (connectionpool:395) "GET /v2/6dfd36b312394e358df805889fb136ee/images/dc323229-2338-41cc-8635-cd701836cf50 HTTP/1.1" 200 679
  DEBUG (session:215) RESP: [200] Content-Type: application/json Content-Length: 679 X-Compute-Request-Id: req-45176905-02e5-4d24-983a-2a4f0df9b1e9 Date: Wed, 25 Jan 2017 17:10:52 GMT Connection: keep-alive
  RESP BODY: {"image": {"status": "ACTIVE", "updated": "2017-01-25T09:41:04Z", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/images/dc323229-2338-41cc-8635-cd701836cf50";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/images/dc323229-2338-41cc-8635-cd701836cf50";, "rel": "bookmark"}, {"href": "http://controller1:9292/images/dc323229-2338-41cc-8635-cd701836cf50";, "type": "application/vnd.openstack.image", "rel": "alternate"}], "id": "dc323229-2338-41cc-8635-cd701836cf50", "OS-EXT-IMG-SIZE:size": 13287936, "name": "cirros", "created": "2017-01-25T09:41:04Z", "minDisk": 0, "progress": 100, "minRam": 0, "metadata": {}}}

  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/m1.tiny -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281"
  DEBUG (connectionpool:395) "GET /v2/6dfd36b312394e358df805889fb136ee/flavors/m1.tiny HTTP/1.1" 404 80
  DEBUG (session:215) RESP: [404] Content-Length: 80 Content-Type: application/json; charset=UTF-8 X-Compute-Request-Id: req-ded67909-b2ff-491e-b682-ab12e56893fb Date: Wed, 25 Jan 2017 17:10:52 GMT Connection: keep-alive
  RESP BODY: {"itemNotFound": {"message": "Flavor m1.tiny could not be found.", "code": 404}}

  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors?is_public=None -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281"
  DEBUG (connectionpool:395) "GET /v2/6dfd36b312394e358df805889fb136ee/flavors?is_public=None HTTP/1.1" 200 1219
  DEBUG (session:215) RESP: [200] Content-Type: application/json Content-Length: 1219 X-Compute-Request-Id: req-5642685a-a188-4d3b-9893-5a5b3845f8b8 Date: Wed, 25 Jan 2017 17:10:52 GMT Connection: keep-alive
  RESP BODY: {"flavors": [{"id": "1", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/1";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/flavors/1";, "rel": "bookmark"}], "name": "m1.tiny"}, {"id": "2", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/2";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/flavors/2";, "rel": "bookmark"}], "name": "m1.small"}, {"id": "3", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/3";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/flavors/3";, "rel": "bookmark"}], "name": "m1.medium"}, {"id": "4", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/4";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/flavors/4";, "rel": "bookmark"}], "name": "m1.large"}, {"id": "5", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/5";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/flavors/5";, "rel": "bookmark"}], "name": "m1.xlarge"}]}

  DEBUG (session:198) REQ: curl -g -i -X GET http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/1 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281"
  DEBUG (connectionpool:395) "GET /v2/6dfd36b312394e358df805889fb136ee/flavors/1 HTTP/1.1" 200 420
  DEBUG (session:215) RESP: [200] Content-Type: application/json Content-Length: 420 X-Compute-Request-Id: req-c2b1ceb1-5c3c-4c24-9d8b-9d9b5da5d274 Date: Wed, 25 Jan 2017 17:10:52 GMT Connection: keep-alive
  RESP BODY: {"flavor": {"name": "m1.tiny", "links": [{"href": "http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/flavors/1";, "rel": "self"}, {"href": "http://controller1:8774/6dfd36b312394e358df805889fb136ee/flavors/1";, "rel": "bookmark"}], "ram": 512, "OS-FLV-DISABLED:disabled": false, "vcpus": 1, "swap": "", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "id": "1"}}

  DEBUG (session:198) REQ: curl -g -i -X POST http://controller1:8774/v2/6dfd36b312394e358df805889fb136ee/servers -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}77756b053b07fc3dbc66433dc91afeab62f11281" -d '{"server": {"name": "private-instance", "imageRef": "dc323229-2338-41cc-8635-cd701836cf50", "key_name": "mykey", "flavorRef": "1", "max_count": 1, "min_count": 1, "networks": [{"uuid": "b4770dbb-b1aa-47a8-ac8c-20bbf528b764"}], "security_groups": [{"name": "default"}]}}'
  DEBUG (connectionpool:395) "POST /v2/6dfd36b312394e358df805889fb136ee/servers HTTP/1.1" 500 216
  DEBUG (session:215) RESP: [500] Content-Length: 216 Content-Type: application/json; charset=UTF-8 X-Compute-Request-Id: req-9ec69d8f-58a5-4310-84f8-253c78c4730b Date: Wed, 25 Jan 2017 17:11:53 GMT Connection: keep-alive
  RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}

  DEBUG (shell:905) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ec69d8f-58a5-4310-84f8-253c78c4730b)
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 903, in main
      OpenStackComputeShell().main(argv)
    File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 830, in main
      args.func(self.cs, args)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 520, in do_boot
      server = cs.servers.create(*boot_args, **boot_kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 942, in create
      **boot_kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 540, in _boot
      return_raw=return_raw, **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 169, in _create
      _resp, body = self.api.client.post(url, body=body)
    File "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 176, in post
      return self.request(url, 'POST', **kwargs)
    File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 93, in request
      raise exceptions.from_response(resp, body, url, method)
  ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ec69d8f-58a5-4310-84f8-253c78c4730b)
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9ec69d8f-58a5-4310-84f8-253c78c4730b)

  Environment
  ======================================
  0.Platform:  Virtualbox
  1.Version:   Liberty
  2.OS:        Centos7.2
  3.Network:   Self-service network
  4.nodes:     controller1 node ,compute1 node(each with 3 nic,one for management with NAT,one for public with bridge,one for xshell with "Host-only" )

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1659366/+subscriptions
References

[Bug 1659366] [NEW] unexpected API error when executing nova boot
From: wu hongjing, 2017-01-25