yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1517224] [NEW] Unable to launch instances via CLI or Horizon

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Dana <dtravers@xxxxxxxxxxxxxx>
Date: Tue, 17 Nov 2015 21:44:20 -0000
Reply-to: Bug 1517224 <1517224@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

I'm unable to launch instances via CLI or Horizon. I can get the same
errors in nova-api.log via CLI or Horizon as well.

I'm running openstack liberty on Ubuntu Server 14.04.03 LTS. All on a
single server.

root@ubuntu:~/openstack# dpkg -l | grep nova
ii  nova-api                             2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - API frontend
ii  nova-cert                            2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - certificate management
ii  nova-common                          2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - common files
ii  nova-compute                         2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - compute node base
ii  nova-compute-kvm                     2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - compute node (KVM)
ii  nova-compute-libvirt                 2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - compute node libvirt support
ii  nova-conductor                       2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - conductor service
ii  nova-consoleauth                     2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - Console Authenticator
ii  nova-novncproxy                      2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - NoVNC proxy
ii  nova-scheduler                       2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - virtual machine scheduler
ii  python-nova                          2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute Python libraries
ii  python-novaclient                    2:2.30.1-1~cloud0                     all          client library for OpenStack Compute API
root@ubuntu:~/openstack#

>From the nova-api.log file:

2015-11-17 21:12:56.080 5593 DEBUG nova.api.openstack.compute.servers [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Running _create_extension_point for <Extension: name=SecurityGroups, alias=os-security-groups, version=1> _create_extension_point /usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py:700
2015-11-17 21:12:56.080 5593 DEBUG nova.api.openstack.compute.servers [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Running _create_extension_point for <Extension: name=ConfigDrive, alias=os-config-drive, version=1> _create_extension_point /usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py:700
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Unexpected exception in API method
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 611, in create
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     **create_kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 149, in inner
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     rv = f(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1581, in create
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     check_server_group_quota=check_server_group_quota)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1181, in _create_instance
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     auto_disk_config, reservation_id, max_count)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 923, in _validate_and_build_base_options
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     requested_networks, max_count)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 533, in _check_requested_networks
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     max_count)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 49, in wrapped
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return func(self, context, *args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 396, in validate_networks
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     requested_networks)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 204, in validate_networks
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return self.client.call(ctxt, 'validate_networks', networks=networks)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 403, in call
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return self.prepare().call(ctxt, method, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     retry=self.retry)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     timeout=timeout, retry=retry)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     retry=retry)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 420, in _send
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     result = self._waiter.wait(msg_id, timeout)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 318, in wait
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     message = self.waiters.get(msg_id, timeout=timeout)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 223, in get
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     'to message ID %s' % msg_id)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions MessagingTimeout: Timed out waiting for a reply to message ID 40ed7a93d4cf436f9445b0b40ba792bb
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions 
2015-11-17 21:13:56.176 5593 INFO nova.api.openstack.wsgi [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'>
2015-11-17 21:13:56.177 5593 DEBUG nova.api.openstack.wsgi [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1175
2015-11-17 21:13:56.177 5593 INFO nova.osapi_compute.wsgi.server [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] 127.0.0.1 "POST /v2/c11ee164bbd64687b3a74332932fb2b2/servers HTTP/1.1" status: 500 len: 447 time: 60.1016970

Steps to reproduce:

# nova --debug boot --flavor m1.tiny --image cirros --nic net-id
=449947fc-2ea4-4460-b360-e2531ed4f1bf --security-group default --key-
name mykey private-instance

INCLUDE PROPER VALUES FOR '--nic net-id=<uuid>' AND '--key-name
<key_name>'

This is the requests that end up with an HTTP 500 response:

DEBUG (session:198) REQ: curl -g -i -X POST
http://localhost:8774/v2/c4d347d211154f2cb5f8248680bbb1e0/servers -H
"User-Agent: python-novaclient" -H "Content-Type: application/json" -H
"Accept: application/json" -H "X-Auth-Token:
{SHA1}3cc67efba6261353fbb66342a38c9a289b81aa2e" -d '{"server": {"name":
"private-instance", "imageRef": "275a16f7-20d3-4dc7-9247-ffce74646dcf",
"key_name": "mykey", "flavorRef": "1", "max_count": 1, "min_count": 1,
"networks": [{"uuid": "449947fc-2ea4-4460-b360-e2531ed4f1bf"}],
"security_groups": [{"name": "default"}]}}'

Here is the response and other debug output:

DEBUG (connectionpool:385) "POST /v2/c4d347d211154f2cb5f8248680bbb1e0/servers HTTP/1.1" 500 216
DEBUG (session:215) RESP: [500] Date: Tue, 17 Nov 2015 21:31:05 GMT Connection: keep-alive Content-Type: application/json; charset=UTF-8 Content-Length: 216 X-Compute-Request-Id: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d 
RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}

DEBUG (shell:905) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 903, in main
    OpenStackComputeShell().main(argv)
  File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 830, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.7/dist-packages/novaclient/v2/shell.py", line 520, in do_boot
    server = cs.servers.create(*boot_args, **boot_kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 942, in create
    **boot_kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 540, in _boot
    return_raw=return_raw, **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in _create
    _resp, body = self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 93, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)

Expected result:

Instance is launched without error.

Actual result:

Instance does not launch and the reported errors are observed in the
nova-api.log file.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1517224

Title:
  Unable to launch instances via CLI or Horizon

Status in OpenStack Compute (nova):
  New

Bug description:
  I'm unable to launch instances via CLI or Horizon. I can get the same
  errors in nova-api.log via CLI or Horizon as well.

  I'm running openstack liberty on Ubuntu Server 14.04.03 LTS. All on a
  single server.

  root@ubuntu:~/openstack# dpkg -l | grep nova
  ii  nova-api                             2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - API frontend
  ii  nova-cert                            2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - certificate management
  ii  nova-common                          2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - common files
  ii  nova-compute                         2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - compute node base
  ii  nova-compute-kvm                     2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - compute node (KVM)
  ii  nova-compute-libvirt                 2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - compute node libvirt support
  ii  nova-conductor                       2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - conductor service
  ii  nova-consoleauth                     2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - Console Authenticator
  ii  nova-novncproxy                      2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - NoVNC proxy
  ii  nova-scheduler                       2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute - virtual machine scheduler
  ii  python-nova                          2:12.0.0-0ubuntu2~cloud0              all          OpenStack Compute Python libraries
  ii  python-novaclient                    2:2.30.1-1~cloud0                     all          client library for OpenStack Compute API
  root@ubuntu:~/openstack#

  From the nova-api.log file:

  2015-11-17 21:12:56.080 5593 DEBUG nova.api.openstack.compute.servers [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Running _create_extension_point for <Extension: name=SecurityGroups, alias=os-security-groups, version=1> _create_extension_point /usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py:700
  2015-11-17 21:12:56.080 5593 DEBUG nova.api.openstack.compute.servers [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Running _create_extension_point for <Extension: name=ConfigDrive, alias=os-config-drive, version=1> _create_extension_point /usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py:700
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Unexpected exception in API method
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions Traceback (most recent call last):
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 611, in create
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     **create_kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 149, in inner
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     rv = f(*args, **kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1581, in create
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     check_server_group_quota=check_server_group_quota)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1181, in _create_instance
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     auto_disk_config, reservation_id, max_count)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 923, in _validate_and_build_base_options
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     requested_networks, max_count)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 533, in _check_requested_networks
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     max_count)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 49, in wrapped
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return func(self, context, *args, **kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 396, in validate_networks
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     requested_networks)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 204, in validate_networks
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return self.client.call(ctxt, 'validate_networks', networks=networks)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 403, in call
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     return self.prepare().call(ctxt, method, **kwargs)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     retry=self.retry)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     timeout=timeout, retry=retry)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     retry=retry)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 420, in _send
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     result = self._waiter.wait(msg_id, timeout)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 318, in wait
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     message = self.waiters.get(msg_id, timeout=timeout)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 223, in get
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions     'to message ID %s' % msg_id)
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions MessagingTimeout: Timed out waiting for a reply to message ID 40ed7a93d4cf436f9445b0b40ba792bb
  2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions 
  2015-11-17 21:13:56.176 5593 INFO nova.api.openstack.wsgi [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'>
  2015-11-17 21:13:56.177 5593 DEBUG nova.api.openstack.wsgi [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1175
  2015-11-17 21:13:56.177 5593 INFO nova.osapi_compute.wsgi.server [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] 127.0.0.1 "POST /v2/c11ee164bbd64687b3a74332932fb2b2/servers HTTP/1.1" status: 500 len: 447 time: 60.1016970

  Steps to reproduce:

  # nova --debug boot --flavor m1.tiny --image cirros --nic net-id
  =449947fc-2ea4-4460-b360-e2531ed4f1bf --security-group default --key-
  name mykey private-instance

  INCLUDE PROPER VALUES FOR '--nic net-id=<uuid>' AND '--key-name
  <key_name>'

  This is the requests that end up with an HTTP 500 response:

  DEBUG (session:198) REQ: curl -g -i -X POST
  http://localhost:8774/v2/c4d347d211154f2cb5f8248680bbb1e0/servers -H
  "User-Agent: python-novaclient" -H "Content-Type: application/json" -H
  "Accept: application/json" -H "X-Auth-Token:
  {SHA1}3cc67efba6261353fbb66342a38c9a289b81aa2e" -d '{"server":
  {"name": "private-instance", "imageRef":
  "275a16f7-20d3-4dc7-9247-ffce74646dcf", "key_name": "mykey",
  "flavorRef": "1", "max_count": 1, "min_count": 1, "networks":
  [{"uuid": "449947fc-2ea4-4460-b360-e2531ed4f1bf"}], "security_groups":
  [{"name": "default"}]}}'

  Here is the response and other debug output:

  DEBUG (connectionpool:385) "POST /v2/c4d347d211154f2cb5f8248680bbb1e0/servers HTTP/1.1" 500 216
  DEBUG (session:215) RESP: [500] Date: Tue, 17 Nov 2015 21:31:05 GMT Connection: keep-alive Content-Type: application/json; charset=UTF-8 Content-Length: 216 X-Compute-Request-Id: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d 
  RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}

  DEBUG (shell:905) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
  Traceback (most recent call last):
    File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 903, in main
      OpenStackComputeShell().main(argv)
    File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 830, in main
      args.func(self.cs, args)
    File "/usr/lib/python2.7/dist-packages/novaclient/v2/shell.py", line 520, in do_boot
      server = cs.servers.create(*boot_args, **boot_kwargs)
    File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 942, in create
      **boot_kwargs)
    File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 540, in _boot
      return_raw=return_raw, **kwargs)
    File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in _create
      _resp, body = self.api.client.post(url, body=body)
    File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
      return self.request(url, 'POST', **kwargs)
    File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 93, in request
      raise exceptions.from_response(resp, body, url, method)
  ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
  ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)

  Expected result:

  Instance is launched without error.

  Actual result:

  Instance does not launch and the reported errors are observed in the
  nova-api.log file.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1517224/+subscriptions
Follow ups

[Bug 1517224] Re: Unable to launch instances via CLI or Horizon
From: Dana, 2015-11-19