yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #41419
[Bug 1517224] Re: Unable to launch instances via CLI or Horizon
Changed status of bug to "Invalid" since this seems to be a
configuration issue.
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1517224
Title:
Unable to launch instances via CLI or Horizon
Status in OpenStack Compute (nova):
Invalid
Bug description:
I'm unable to launch instances via CLI or Horizon. I can get the same
errors in nova-api.log via CLI or Horizon as well.
I'm running openstack liberty on Ubuntu Server 14.04.03 LTS. All on a
single server.
root@ubuntu:~/openstack# dpkg -l | grep nova
ii nova-api 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - API frontend
ii nova-cert 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - certificate management
ii nova-common 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - common files
ii nova-compute 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node base
ii nova-compute-kvm 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node (KVM)
ii nova-compute-libvirt 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node libvirt support
ii nova-conductor 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - conductor service
ii nova-consoleauth 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - Console Authenticator
ii nova-novncproxy 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - NoVNC proxy
ii nova-scheduler 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute - virtual machine scheduler
ii python-nova 2:12.0.0-0ubuntu2~cloud0 all OpenStack Compute Python libraries
ii python-novaclient 2:2.30.1-1~cloud0 all client library for OpenStack Compute API
root@ubuntu:~/openstack#
From the nova-api.log file:
2015-11-17 21:12:56.080 5593 DEBUG nova.api.openstack.compute.servers [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Running _create_extension_point for <Extension: name=SecurityGroups, alias=os-security-groups, version=1> _create_extension_point /usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py:700
2015-11-17 21:12:56.080 5593 DEBUG nova.api.openstack.compute.servers [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Running _create_extension_point for <Extension: name=ConfigDrive, alias=os-config-drive, version=1> _create_extension_point /usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py:700
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Unexpected exception in API method
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions return f(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 611, in create
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions **create_kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 149, in inner
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions rv = f(*args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1581, in create
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions check_server_group_quota=check_server_group_quota)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1181, in _create_instance
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions auto_disk_config, reservation_id, max_count)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 923, in _validate_and_build_base_options
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions requested_networks, max_count)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 533, in _check_requested_networks
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions max_count)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 49, in wrapped
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions return func(self, context, *args, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 396, in validate_networks
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions requested_networks)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 204, in validate_networks
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions return self.client.call(ctxt, 'validate_networks', networks=networks)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 403, in call
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions return self.prepare().call(ctxt, method, **kwargs)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions retry=self.retry)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions timeout=timeout, retry=retry)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 431, in send
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions retry=retry)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 420, in _send
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions result = self._waiter.wait(msg_id, timeout)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 318, in wait
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions message = self.waiters.get(msg_id, timeout=timeout)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 223, in get
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions 'to message ID %s' % msg_id)
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions MessagingTimeout: Timed out waiting for a reply to message ID 40ed7a93d4cf436f9445b0b40ba792bb
2015-11-17 21:13:56.175 5593 ERROR nova.api.openstack.extensions
2015-11-17 21:13:56.176 5593 INFO nova.api.openstack.wsgi [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'>
2015-11-17 21:13:56.177 5593 DEBUG nova.api.openstack.wsgi [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1175
2015-11-17 21:13:56.177 5593 INFO nova.osapi_compute.wsgi.server [req-f428c8c4-56ba-4b8f-970d-84041ebe50db 2a6bd03d76754e35af693c2f24a18ecd c11ee164bbd64687b3a74332932fb2b2 - - -] 127.0.0.1 "POST /v2/c11ee164bbd64687b3a74332932fb2b2/servers HTTP/1.1" status: 500 len: 447 time: 60.1016970
Steps to reproduce:
# nova --debug boot --flavor m1.tiny --image cirros --nic net-id
=449947fc-2ea4-4460-b360-e2531ed4f1bf --security-group default --key-
name mykey private-instance
INCLUDE PROPER VALUES FOR '--nic net-id=<uuid>' AND '--key-name
<key_name>'
This is the requests that end up with an HTTP 500 response:
DEBUG (session:198) REQ: curl -g -i -X POST
http://localhost:8774/v2/c4d347d211154f2cb5f8248680bbb1e0/servers -H
"User-Agent: python-novaclient" -H "Content-Type: application/json" -H
"Accept: application/json" -H "X-Auth-Token:
{SHA1}3cc67efba6261353fbb66342a38c9a289b81aa2e" -d '{"server":
{"name": "private-instance", "imageRef":
"275a16f7-20d3-4dc7-9247-ffce74646dcf", "key_name": "mykey",
"flavorRef": "1", "max_count": 1, "min_count": 1, "networks":
[{"uuid": "449947fc-2ea4-4460-b360-e2531ed4f1bf"}], "security_groups":
[{"name": "default"}]}}'
Here is the response and other debug output:
DEBUG (connectionpool:385) "POST /v2/c4d347d211154f2cb5f8248680bbb1e0/servers HTTP/1.1" 500 216
DEBUG (session:215) RESP: [500] Date: Tue, 17 Nov 2015 21:31:05 GMT Connection: keep-alive Content-Type: application/json; charset=UTF-8 Content-Length: 216 X-Compute-Request-Id: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d
RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}
DEBUG (shell:905) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 903, in main
OpenStackComputeShell().main(argv)
File "/usr/lib/python2.7/dist-packages/novaclient/shell.py", line 830, in main
args.func(self.cs, args)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/shell.py", line 520, in do_boot
server = cs.servers.create(*boot_args, **boot_kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 942, in create
**boot_kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 540, in _boot
return_raw=return_raw, **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 169, in _create
_resp, body = self.api.client.post(url, body=body)
File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line 176, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 93, in request
raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-b92cdadb-d135-4099-bdb6-d41218c1cb6d)
Expected result:
Instance is launched without error.
Actual result:
Instance does not launch and the reported errors are observed in the
nova-api.log file.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1517224/+subscriptions
References
