yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #49972
[Bug 1575154] Re: Ubuntu 16.04, Unexpected API Error, , Nova
Closed as requested in comment #2.
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1575154
Title:
Ubuntu 16.04, Unexpected API Error,, Nova
Status in OpenStack Compute (nova):
Invalid
Bug description:
Creating an instance fails:
Command: openstack server create --flavor 42b791ba-3330-4a8c-b8bc-5b8afe94b7ce --image 6b5bfd61-8c35-44e0-be49-da12a6988cd1 --nic net-id=16ed9203-beaf-42f0-bf37-b7bdf68839f1 --security-group default --key-name testinstance public-instance --debug
(I'm following: http://docs.openstack.org/liberty/install-guide-ubuntu
/launch-instance-public.html, all commands nova flavor-list, nova
image-list, .... work without issues)
....
boot_args: ['public-instance', <Image: xenial>, <Flavor: m1.small>]
boot_kwargs: {'files': {}, 'userdata': None, 'availability_zone': None, 'nics': [{'port-id': '', 'net-id': u'16ed9203-beaf-42f0-bf37-b7bdf68839f1', 'v4-fixed-ip': '', 'v6-fixed-ip': ''}], 'block_device_mapping': {}, 'max_count': 1, 'meta': None, 'key_name': 'testinstance', 'min_count': 1, 'scheduler_hints': {}, 'reservation_id': None, 'security_groups': ['default'], 'config_drive': None}
REQ: curl -g -i -X POST http://172.24.33.142:8774/v2.1/172f573ed78a4fc7b3460ea4a3ac6dbb/servers -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}fadf0d364f2eb45845658431ef42d15ddf1d4e8a" -d '{"server": {"name": "public-instance", "imageRef": "6b5bfd61-8c35-44e0-be49-da12a6988cd1", "key_name": "testinstance", "flavorRef": "42b791ba-3330-4a8c-b8bc-5b8afe94b7ce", "max_count": 1, "min_count": 1, "networks": [{"uuid": "16ed9203-beaf-42f0-bf37-b7bdf68839f1"}], "security_groups": [{"name": "default"}]}}'
"POST /v2.1/172f573ed78a4fc7b3460ea4a3ac6dbb/servers HTTP/1.1" 500 216
RESP: [500] Content-Length: 216 X-Compute-Request-Id: req-9bc78003-48a4-4059-9dad-ed7ee468c931 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.1 Date: Tue, 26 Apr 2016 12:01:13 GMT Content-Type: application/json; charset=UTF-8
RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_messaging.exceptions.MessagingTimeout'>", "code": 500}}
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9bc78003-48a4-4059-9dad-ed7ee468c931)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 374, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/dist-packages/openstackclient/common/command.py", line 38, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 92, in run
column_names, data = self.take_action(parsed_args)
File "/usr/lib/python2.7/dist-packages/openstackclient/compute/v2/server.py", line 519, in take_action
server = compute_client.servers.create(*boot_args, **boot_kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 1233, in create
**boot_kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 667, in _boot
return_raw=return_raw, **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 345, in _create
resp, body = self.api.client.post(url, body=body)
File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 179, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 94, in request
raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9bc78003-48a4-4059-9dad-ed7ee468c931)
clean_up CreateServer: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9bc78003-48a4-4059-9dad-ed7ee468c931)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/openstackclient/shell.py", line 118, in run
ret_val = super(OpenStackShell, self).run(argv)
File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 255, in run
result = self.run_subcommand(remainder)
File "/usr/lib/python2.7/dist-packages/openstackclient/shell.py", line 153, in run_subcommand
ret_value = super(OpenStackShell, self).run_subcommand(argv)
File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 374, in run_subcommand
result = cmd.run(parsed_args)
File "/usr/lib/python2.7/dist-packages/openstackclient/common/command.py", line 38, in run
return super(Command, self).run(parsed_args)
File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 92, in run
column_names, data = self.take_action(parsed_args)
File "/usr/lib/python2.7/dist-packages/openstackclient/compute/v2/server.py", line 519, in take_action
server = compute_client.servers.create(*boot_args, **boot_kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 1233, in create
**boot_kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/v2/servers.py", line 667, in _boot
return_raw=return_raw, **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/base.py", line 345, in _create
resp, body = self.api.client.post(url, body=body)
File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 179, in post
return self.request(url, 'POST', **kwargs)
File "/usr/lib/python2.7/dist-packages/novaclient/client.py", line 94, in request
raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> (HTTP 500) (Request-ID: req-9bc78003-48a4-4059-9dad-ed7ee468c931)
END return value: 1
Version:
dpkg -l | grep nova [0]
ii nova-api 2:13.0.0-0ubuntu2 all OpenStack Compute - API frontend
ii nova-common 2:13.0.0-0ubuntu2 all OpenStack Compute - common files
ii nova-conductor 2:13.0.0-0ubuntu2 all OpenStack Compute - conductor service
ii nova-consoleauth 2:13.0.0-0ubuntu2 all OpenStack Compute - Console Authenticator
ii nova-novncproxy 2:13.0.0-0ubuntu2 all OpenStack Compute - NoVNC proxy
ii nova-scheduler 2:13.0.0-0ubuntu2 all OpenStack Compute - virtual machine scheduler
ii python-nova 2:13.0.0-0ubuntu2 all OpenStack Compute Python libraries
ii python-novaclient 2:3.3.1-2 all client library for OpenStack Compute API - Python 2.7
nova.log
....
2016-04-26 14:00:38.830 11632 DEBUG nova.osapi_compute.wsgi.server [req-946a4436-b649-4e3e-87ed-8731cebdf996 - - - - -] (11632) accepted ('172.24.33.142', 53990) server /usr/lib/python2.7/dist-packages/eventlet/wsgi.py:867
2016-04-26 14:00:38.840 11632 DEBUG oslo_policy._cache_handler [req-9af31c64-0495-452b-b5a8-1a6e7543adc4 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Reloading cached file /etc/nova/policy.json read_cached_file /usr/lib/python2.7/dist-packages/oslo_policy/_cache_handler.py:38
2016-04-26 14:00:38.853 11632 DEBUG oslo_policy.policy [req-9af31c64-0495-452b-b5a8-1a6e7543adc4 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Reloaded policy file: /etc/nova/policy.json _load_policy_file /usr/lib/python2.7/dist-packages/oslo_policy/policy.py:493
2016-04-26 14:00:39.470 11632 DEBUG nova.api.openstack.wsgi [req-9af31c64-0495-452b-b5a8-1a6e7543adc4 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x7fd3fc901a10>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:699
2016-04-26 14:00:39.471 11632 DEBUG nova.compute.api [req-9af31c64-0495-452b-b5a8-1a6e7543adc4 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Searching by: {'deleted': False, u'project_id': u'7f96e108b48e4f6dbb78aa05820891f2'} get_all /usr/lib/python2.7/dist-packages/nova/compute/api.py:2055
2016-04-26 14:00:39.502 11632 DEBUG nova.policy [req-9af31c64-0495-452b-b5a8-1a6e7543adc4 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Policy check for os_compute_api:os-hide-server-addresses failed with credentials {'domain': None, 'project_name': u'admin', 'project_domain': None, 'timestamp': '2016-04-26T12:00:38.839620', 'remote_address': '172.24.33.142', 'quota_class': None, 'resource_uuid': None, 'is_admin': True, 'user': u'20a76a29d52f4ef5a9965cfd1478a73b', 'service_catalog': [], 'tenant': u'7f96e108b48e4f6dbb78aa05820891f2', 'read_only': False, 'project_id': u'7f96e108b48e4f6dbb78aa05820891f2', 'user_id': u'20a76a29d52f4ef5a9965cfd1478a73b', 'show_deleted': False, 'roles': [u'admin'], 'user_identity': '20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -', 'read_deleted': 'no', 'request_id': 'req-9af31c64-0495-452b-b5a8-1a6e7543adc4', 'instance_lock_checked': False, 'user_domain': None, 'user_name': u'admin'} enforce /usr/lib/python2.7/dist-packages/nova/policy.py:107
2016-04-26 14:00:39.504 11632 INFO nova.osapi_compute.wsgi.server [req-9af31c64-0495-452b-b5a8-1a6e7543adc4 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] 172.24.33.142 "GET /v2.1/7f96e108b48e4f6dbb78aa05820891f2/servers/detail?project_id=7f96e108b48e4f6dbb78aa05820891f2 HTTP/1.1" status: 200 len: 282 time: 0.6717088
2016-04-26 14:00:49.828 11618 DEBUG nova.osapi_compute.wsgi.server [req-8c1e9cbc-53d4-40e9-976b-f39c9dea0d60 - - - - -] (11618) accepted ('172.24.33.142', 54014) server /usr/lib/python2.7/dist-packages/eventlet/wsgi.py:867
2016-04-26 14:00:49.838 11618 DEBUG oslo_policy._cache_handler [req-0b0ee67b-08f3-4120-9c87-c161474e549a 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Reloading cached file /etc/nova/policy.json read_cached_file /usr/lib/python2.7/dist-packages/oslo_policy/_cache_handler.py:38
2016-04-26 14:00:49.851 11618 DEBUG oslo_policy.policy [req-0b0ee67b-08f3-4120-9c87-c161474e549a 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Reloaded policy file: /etc/nova/policy.json _load_policy_file /usr/lib/python2.7/dist-packages/oslo_policy/policy.py:493
2016-04-26 14:00:50.438 11618 DEBUG nova.api.openstack.wsgi [req-0b0ee67b-08f3-4120-9c87-c161474e549a 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Calling method '<bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x7fd3fc901a10>>' _process_stack /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:699
2016-04-26 14:00:50.439 11618 DEBUG nova.compute.api [req-0b0ee67b-08f3-4120-9c87-c161474e549a 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Searching by: {'deleted': False, u'project_id': u'7f96e108b48e4f6dbb78aa05820891f2'} get_all /usr/lib/python2.7/dist-packages/nova/compute/api.py:2055
2016-04-26 14:00:50.470 11618 DEBUG nova.policy [req-0b0ee67b-08f3-4120-9c87-c161474e549a 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] Policy check for os_compute_api:os-hide-server-addresses failed with credentials {'domain': None, 'project_name': u'admin', 'project_domain': None, 'timestamp': '2016-04-26T12:00:49.837449', 'remote_address': '172.24.33.142', 'quota_class': None, 'resource_uuid': None, 'is_admin': True, 'user': u'20a76a29d52f4ef5a9965cfd1478a73b', 'service_catalog': [], 'tenant': u'7f96e108b48e4f6dbb78aa05820891f2', 'read_only': False, 'project_id': u'7f96e108b48e4f6dbb78aa05820891f2', 'user_id': u'20a76a29d52f4ef5a9965cfd1478a73b', 'show_deleted': False, 'roles': [u'admin'], 'user_identity': '20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -', 'read_deleted': 'no', 'request_id': 'req-0b0ee67b-08f3-4120-9c87-c161474e549a', 'instance_lock_checked': False, 'user_domain': None, 'user_name': u'admin'} enforce /usr/lib/python2.7/dist-packages/nova/policy.py:107
2016-04-26 14:00:50.472 11618 INFO nova.osapi_compute.wsgi.server [req-0b0ee67b-08f3-4120-9c87-c161474e549a 20a76a29d52f4ef5a9965cfd1478a73b 7f96e108b48e4f6dbb78aa05820891f2 - - -] 172.24.33.142 "GET /v2.1/7f96e108b48e4f6dbb78aa05820891f2/servers/detail?project_id=7f96e108b48e4f6dbb78aa05820891f2 HTTP/1.1" status: 200 len: 282 time: 0.6421530
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions [req-9bc78003-48a4-4059-9dad-ed7ee468c931 6d57698fd7114db0b2151744cee0bc49 172f573ed78a4fc7b3460ea4a3ac6dbb - - -] Unexpected exception in API method
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return f(*args, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return func(*args, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 629, in create
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions **create_kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/hooks.py", line 154, in inner
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions rv = f(*args, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1556, in create
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions check_server_group_quota=check_server_group_quota)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 1139, in _create_instance
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions reservation_id, max_count)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 834, in _validate_and_build_base_options
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions requested_networks, max_count)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 448, in _check_requested_networks
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions max_count)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 49, in wrapped
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return func(self, context, *args, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 399, in validate_networks
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions requested_networks)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 212, in validate_networks
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return self.client.call(ctxt, 'validate_networks', networks=networks)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 413, in call
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions return self.prepare().call(ctxt, method, **kwargs)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions retry=self.retry)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions timeout=timeout, retry=retry)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 470, in send
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions retry=retry)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 459, in _send
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions result = self._waiter.wait(msg_id, timeout)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 342, in wait
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions message = self.waiters.get(msg_id, timeout=timeout)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 244, in get
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions 'to message ID %s' % msg_id)
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions MessagingTimeout: Timed out waiting for a reply to message ID aa87e40c809840dead096c6644f66ba7
2016-04-26 14:01:13.865 11628 ERROR nova.api.openstack.extensions
2016-04-26 14:01:13.871 11628 INFO nova.api.openstack.wsgi [req-9bc78003-48a4-4059-9dad-ed7ee468c931 6d57698fd7114db0b2151744cee0bc49 172f573ed78a4fc7b3460ea4a3ac6dbb - - -] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'>
2016-04-26 14:01:13.873 11628 DEBUG nova.api.openstack.wsgi [req-9bc78003-48a4-4059-9dad-ed7ee468c931 6d57698fd7114db0b2151744cee0bc49 172f573ed78a4fc7b3460ea4a3ac6dbb - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'oslo_messaging.exceptions.MessagingTimeout'> __call__ /usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py:1070
2016-04-26 14:01:13.875 11628 INFO nova.osapi_compute.wsgi.server [req-9bc78003-48a4-4059-9dad-ed7ee468c931 6d57698fd7114db0b2151744cee0bc49 172f573ed78a4fc7b3460ea4a3ac6dbb - - -] 172.24.33.142 "POST /v2.1/172f573ed78a4fc7b3460ea4a3ac6dbb/servers HTTP/1.1" status: 500 len: 518 time: 60.1486168
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1575154/+subscriptions
References