yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1701174] [NEW] ERROR nova.api, Unexpected API Error

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: zeta <1701174@xxxxxxxxxxxxxxxxxx>
Date: Thu, 29 Jun 2017 05:52:14 -0000
Reply-to: Bug 1701174 <1701174@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 338, in wrapped
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 207, in detail
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     servers = self._get_servers(req, is_detail=True)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 347, in _get_servers
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 2330, in get_all
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 2407, in _get_instances_by_filters
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     expected_attrs=fields, sort_keys=sort_keys, sort_dirs=sort_dirs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     result = fn(cls, context, *args, **kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1198, in get_by_filters
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     use_slave=use_slave, sort_keys=sort_keys, sort_dirs=sort_dirs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 225, in wrapper
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1182, in _get_by_filters_impl
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 763, in instance_get_all_by_filters_sort
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 169, in wrapper
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 269, in wrapped
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     with ctxt_mgr.reader.allow_async.using(context):
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self.gen.next()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 874, in _transaction_scope
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     allow_async=self._allow_async) as resource:
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self.gen.next()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 518, in _session
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     bind=self.connection, mode=self.mode)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 277, in _create_session
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self._start()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 363, in _start
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     engine_args, maker_args)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 387, in _setup_for_connection
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sql_connection=sql_connection, **engine_kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 155, in create_engine
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     test_conn = _test_connection(engine, max_retries, retry_interval)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 331, in _test_connection
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return engine.connect()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2018, in connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self._connection_cls(self, **kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 72, in __init__
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     if connection is not None else engine.raw_connection()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2104, in raw_connection
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self.pool.unique_connection, _connection)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2078, in _wrap_pool_connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     e, dialect, self)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1401, in _handle_dbapi_exception_noconnection
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     util.raise_from_cause(newraise, exc_info)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     reraise(type(exception), exception, tb=exc_tb)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2074, in _wrap_pool_connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return fn()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 318, in unique_connection
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return _ConnectionFairy._checkout(self)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 713, in _checkout
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     fairy = _ConnectionRecord.checkout(pool)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 480, in checkout
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     rec = pool._do_get()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1060, in _do_get
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self._dec_overflow()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     compat.reraise(exc_type, exc_value, exc_tb)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1057, in _do_get
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self._create_connection()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 323, in _create_connection
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return _ConnectionRecord(self)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 449, in __init__
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self.connection = self.__connect()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 607, in __connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     connection = self.__pool._invoke_creator(self)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 97, in connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return dialect.connect(*cargs, **cparams)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 385, in connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self.dbapi.connect(*cargs, **cparams)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/__init__.py", line 88, in Connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return Connection(*args, **kwargs)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 679, in __init__
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self.connect()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 891, in connect
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self._request_authentication()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1097, in _request_authentication
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     auth_packet = self._read_packet()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 966, in _read_packet
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     packet.check_error()
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 394, in check_error
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     err.raise_mysql_exception(self._data)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     _check_mysql_exception(errinfo)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     raise errorclass(errno, errorvalue)
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions OperationalError: (pymysql.err.OperationalError) (1044, u"Access denied for user 'nova'@'%' to database 'nova_api_cell0'")
2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions
2017-06-29 12:25:13.483 2629 INFO nova.api.openstack.wsgi [req-667d9bbc-b45d-483e-8de8-2560a5ad93ab a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
2017-06-29 12:25:13.707 2629 INFO nova.osapi_compute.wsgi.server [req-667d9bbc-b45d-483e-8de8-2560a5ad93ab a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/883f2863818b411a8c86fbbcdc8f3232/servers/detail HTTP/1.1" status: 500 len: 574 time: 4.8026721
2017-06-29 12:26:50.939 2628 INFO nova.osapi_compute.wsgi.server [req-8a67b8f6-915c-417e-a503-88c94f1fb447 a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/883f2863818b411a8c86fbbcdc8f3232 HTTP/1.1" status: 404 len: 331 time: 3.8801279
2017-06-29 12:26:53.375 2628 INFO nova.osapi_compute.wsgi.server [req-2491dbc8-5946-44f8-905e-1ee5a7fea6f1 a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/ HTTP/1.1" status: 200 len: 718 time: 2.3365922
2017-06-29 12:27:00.273 2628 INFO nova.osapi_compute.wsgi.server [req-6427d289-326b-4f3a-ab33-7a295c11b94f a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/883f2863818b411a8c86fbbcdc8f3232/os-services HTTP/1.1" status: 200 len: 1181 time: 6.6650732

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1701174

Title:
   ERROR nova.api, Unexpected API Error

Status in OpenStack Compute (nova):
  New

Bug description:
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions Traceback (most recent call last):
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/extensions.py", line 338, in wrapped
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 207, in detail
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     servers = self._get_servers(req, is_detail=True)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/api/openstack/compute/servers.py", line 347, in _get_servers
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 2330, in get_all
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/compute/api.py", line 2407, in _get_instances_by_filters
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     expected_attrs=fields, sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     result = fn(cls, context, *args, **kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1198, in get_by_filters
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     use_slave=use_slave, sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 225, in wrapper
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 1182, in _get_by_filters_impl
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 763, in instance_get_all_by_filters_sort
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 169, in wrapper
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 269, in wrapped
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     with ctxt_mgr.reader.allow_async.using(context):
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self.gen.next()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 874, in _transaction_scope
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     allow_async=self._allow_async) as resource:
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self.gen.next()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 518, in _session
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     bind=self.connection, mode=self.mode)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 277, in _create_session
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self._start()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 363, in _start
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     engine_args, maker_args)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 387, in _setup_for_connection
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     sql_connection=sql_connection, **engine_kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 155, in create_engine
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     test_conn = _test_connection(engine, max_retries, retry_interval)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line 331, in _test_connection
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return engine.connect()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2018, in connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self._connection_cls(self, **kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 72, in __init__
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     if connection is not None else engine.raw_connection()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2104, in raw_connection
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self.pool.unique_connection, _connection)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2078, in _wrap_pool_connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     e, dialect, self)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1401, in _handle_dbapi_exception_noconnection
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     util.raise_from_cause(newraise, exc_info)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     reraise(type(exception), exception, tb=exc_tb)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2074, in _wrap_pool_connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return fn()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 318, in unique_connection
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return _ConnectionFairy._checkout(self)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 713, in _checkout
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     fairy = _ConnectionRecord.checkout(pool)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 480, in checkout
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     rec = pool._do_get()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1060, in _do_get
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self._dec_overflow()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     compat.reraise(exc_type, exc_value, exc_tb)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 1057, in _do_get
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self._create_connection()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 323, in _create_connection
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return _ConnectionRecord(self)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 449, in __init__
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self.connection = self.__connect()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 607, in __connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     connection = self.__pool._invoke_creator(self)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 97, in connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return dialect.connect(*cargs, **cparams)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 385, in connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return self.dbapi.connect(*cargs, **cparams)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/__init__.py", line 88, in Connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     return Connection(*args, **kwargs)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 679, in __init__
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self.connect()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 891, in connect
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     self._request_authentication()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1097, in _request_authentication
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     auth_packet = self._read_packet()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 966, in _read_packet
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     packet.check_error()
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 394, in check_error
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     err.raise_mysql_exception(self._data)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 120, in raise_mysql_exception
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     _check_mysql_exception(errinfo)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 112, in _check_mysql_exception
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions     raise errorclass(errno, errorvalue)
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions OperationalError: (pymysql.err.OperationalError) (1044, u"Access denied for user 'nova'@'%' to database 'nova_api_cell0'")
  2017-06-29 12:25:12.680 2629 ERROR nova.api.openstack.extensions
  2017-06-29 12:25:13.483 2629 INFO nova.api.openstack.wsgi [req-667d9bbc-b45d-483e-8de8-2560a5ad93ab a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
  2017-06-29 12:25:13.707 2629 INFO nova.osapi_compute.wsgi.server [req-667d9bbc-b45d-483e-8de8-2560a5ad93ab a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/883f2863818b411a8c86fbbcdc8f3232/servers/detail HTTP/1.1" status: 500 len: 574 time: 4.8026721
  2017-06-29 12:26:50.939 2628 INFO nova.osapi_compute.wsgi.server [req-8a67b8f6-915c-417e-a503-88c94f1fb447 a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/883f2863818b411a8c86fbbcdc8f3232 HTTP/1.1" status: 404 len: 331 time: 3.8801279
  2017-06-29 12:26:53.375 2628 INFO nova.osapi_compute.wsgi.server [req-2491dbc8-5946-44f8-905e-1ee5a7fea6f1 a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/ HTTP/1.1" status: 200 len: 718 time: 2.3365922
  2017-06-29 12:27:00.273 2628 INFO nova.osapi_compute.wsgi.server [req-6427d289-326b-4f3a-ab33-7a295c11b94f a50b02c18f564288944e27579926fdaa 883f2863818b411a8c86fbbcdc8f3232 - default default] 10.10.190.108 "GET /v2.1/883f2863818b411a8c86fbbcdc8f3232/os-services HTTP/1.1" status: 200 len: 1181 time: 6.6650732

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1701174/+subscriptions
Follow ups

[Bug 1701174] Re: ERROR nova.api, Unexpected API Error
From: Alex Xu, 2017-06-29