yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1789382] [NEW] openstack server list error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: cc <1789382@xxxxxxxxxxxxxxxxxx>
Date: Tue, 28 Aug 2018 08:17:03 -0000
Reply-to: Bug 1789382 <1789382@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

when execute openstack server list,Show the following error:
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-6a17a629-6bc5-4297-90b4-524dd96b5694)
everytime I see this in nova-api.log:

2018-08-28 16:01:12.660 1092436 WARNING nova.context [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Timed out waiting for response from cell 4d2b47d3-7d78-4061-97fe-89c32e878fa7: CellTimeout: Timeout waiting for response from cell
2018-08-28 16:01:12.661 1092436 WARNING nova.context [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Timed out waiting for response from cell 00000000-0000-0000-0000-000000000000: CellTimeout: Timeout waiting for response from cell
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Unexpected exception in API method: TypeError: 'object' object is not iterable
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 165, in detail
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     servers = self._get_servers(req, is_detail=True)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 309, in _get_servers
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     sort_keys=sort_keys, sort_dirs=sort_dirs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2500, in get_all
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     context, filters, limit, marker, fields, sort_keys, sort_dirs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/compute/instance_list.py", line 107, in get_instance_objects_sorted
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     expected_attrs)
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1208, in _make_instance_list
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     for db_inst in db_inst_list:
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/compute/multi_cell_list.py", line 259, in get_records_sorted
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     for i in heapq.merge(*results.values()):
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib64/python2.7/heapq.py", line 372, in merge
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     for itnum, it in enumerate(map(iter, iterables)):
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi TypeError: 'object' object is not iterable
2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi
2018-08-28 16:01:12.702 1092436 INFO nova.api.openstack.wsgi [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'>
2018-08-28 16:01:12.705 1092436 INFO nova.osapi_compute.wsgi.server [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] 18.5.253.11 "GET /v2.1/servers/detail HTTP/1.1" status: 500 len: 626 time: 120.9522789

When I log in to the dashboard, I have to wait a long time and report the same error.
I have checked the database configuration of nova.conf。

if anyone have or had this problem and can propose a fix i would appreciate it, thanks in advance.
Note: i could provide anything if needed

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1789382

Title:
  openstack server list error

Status in OpenStack Compute (nova):
  New

Bug description:
  when execute openstack server list,Show the following error:
  Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-6a17a629-6bc5-4297-90b4-524dd96b5694)
  everytime I see this in nova-api.log:

  2018-08-28 16:01:12.660 1092436 WARNING nova.context [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Timed out waiting for response from cell 4d2b47d3-7d78-4061-97fe-89c32e878fa7: CellTimeout: Timeout waiting for response from cell
  2018-08-28 16:01:12.661 1092436 WARNING nova.context [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Timed out waiting for response from cell 00000000-0000-0000-0000-000000000000: CellTimeout: Timeout waiting for response from cell
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] Unexpected exception in API method: TypeError: 'object' object is not iterable
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi Traceback (most recent call last):
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     return f(*args, **kwargs)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     return func(*args, **kwargs)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 165, in detail
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     servers = self._get_servers(req, is_detail=True)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 309, in _get_servers
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     sort_keys=sort_keys, sort_dirs=sort_dirs)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2500, in get_all
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     context, filters, limit, marker, fields, sort_keys, sort_dirs)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/compute/instance_list.py", line 107, in get_instance_objects_sorted
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     expected_attrs)
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1208, in _make_instance_list
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     for db_inst in db_inst_list:
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib/python2.7/site-packages/nova/compute/multi_cell_list.py", line 259, in get_records_sorted
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     for i in heapq.merge(*results.values()):
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi   File "/usr/lib64/python2.7/heapq.py", line 372, in merge
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi     for itnum, it in enumerate(map(iter, iterables)):
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi TypeError: 'object' object is not iterable
  2018-08-28 16:01:12.662 1092436 ERROR nova.api.openstack.wsgi
  2018-08-28 16:01:12.702 1092436 INFO nova.api.openstack.wsgi [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  <type 'exceptions.TypeError'>
  2018-08-28 16:01:12.705 1092436 INFO nova.osapi_compute.wsgi.server [req-6a17a629-6bc5-4297-90b4-524dd96b5694 b0dc354f61bb4c88ab5c544079f98c2b cbe025541b814d42ada3028732353516 - default default] 18.5.253.11 "GET /v2.1/servers/detail HTTP/1.1" status: 500 len: 626 time: 120.9522789

  When I log in to the dashboard, I have to wait a long time and report the same error.
  I have checked the database configuration of nova.conf。

  if anyone have or had this problem and can propose a fix i would appreciate it, thanks in advance.
  Note: i could provide anything if needed

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1789382/+subscriptions
Follow ups

[Bug 1789382] Re: openstack server list error
From: Launchpad Bug Tracker, 2018-12-17