yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1915143] [NEW] Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-538dfec9-8464-4a92-9b03-2c610c3eff07)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sanel Tuscano <1915143@xxxxxxxxxxxxxxxxxx>
Date: Tue, 09 Feb 2021 11:25:07 -0000
Reply-to: Bug 1915143 <1915143@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Description
=================================================

I am logged in as admin and created some instances and images as well ,
they were visible for one week but suddenly when I tried to check today
I wasnt able to see any images or instances or volumes , however i was
able to see the networks I created and the routers.

The error I receved was:

Error: Unable to retrieve usage information.
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-538dfec9-8464-4a92-9b03-2c610c3eff07)

Environment
================================================
Openstack Ussuri Release

The log file

=============================================

2021-02-09 11:43:02.838 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:43:02.843 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:43:02.848 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:43:03.887 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] [cad79092-0027-438d-9cce-8fbe181f4ff3] Reconnected to AMQP server on 172.25.8.252:5672 via [amqp] client with port 53754.
2021-02-09 11:43:04.573 1506671 INFO nova.api.openstack.requestlog [req-d2ca1be8-7bb3-45d1-91e2-0bf598112be7 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08/servers/detail?project_id=efc6ffe3908d447987ba5ed6d798ae08" status: 200 len: 17406 microversion: 2.42 time: 1.737148
2021-02-09 11:43:05.501 1506667 INFO nova.api.openstack.requestlog [req-02697d51-58f8-4528-afe2-2d24d8e802a5 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08" status: 404 len: 112 microversion: - time: 0.186829
2021-02-09 11:43:05.510 1506670 INFO nova.api.openstack.requestlog [req-2fd6b62e-cb1f-4919-88d2-d41af9b3a7b9 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/" status: 200 len: 386 microversion: 2.1 time: 0.006624
2021-02-09 11:43:06.220 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:43:06.231 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:43:06.235 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:43:06.737 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] [67d970a3-15cd-4b87-8b2d-a47bd906b100] Reconnected to AMQP server on 172.25.8.252:5672 via [amqp] client with port 53794.
2021-02-09 11:43:08.143 1506666 INFO nova.api.openstack.requestlog [req-11e3f941-ecee-45e7-9964-d70b063a0290 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08/servers/detail?project_id=efc6ffe3908d447987ba5ed6d798ae08" status: 200 len: 17406 microversion: 2.42 time: 2.034052
2021-02-09 11:48:43.174 1506669 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 11:48:43.178 1506669 ERROR oslo.messaging._drivers.impl_rabbit [-] [7916d861-ee6d-484c-9cab-1cd4946ec11d] AMQP server on 172.25.8.252:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer
2021-02-09 11:48:43.388 1506669 INFO nova.api.openstack.requestlog [req-8b40bd1f-371b-40a2-aa78-6f073b80342a 12d3b5fee3be4e9c94bb60c8acc14997 781dc171cf7642dfa11c6649989fcd2d - default default] 172.25.8.252 "GET /v2.1/781dc171cf7642dfa11c6649989fcd2d/servers/detail?all_tenants=True&changes-since=2021-02-09T06%3A08%3A43.495476%2B00%3A00" status: 200 len: 15 microversion: 2.1 time: 0.216355
2021-02-09 14:34:31.067 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 14:34:31.119 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 14:34:31.124 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
2021-02-09 14:34:31.130 1506667 ERROR oslo.messaging._drivers.impl_rabbit [-] [eea1f819-0d22-4034-bbfa-83b93a9be40e] AMQP server on 172.25.8.252:5672 is unreachable: [Errno 104

** Affects: nova
     Importance: Undecided
         Status: New

** Description changed:

+ Description
+ =================================================
+ 
  I am logged in as admin and created some instances and images as well ,
  they were visible for one week but suddenly when I tried to check today
  I wasnt able to see any images or instances or volumes , however i was
  able to see the networks I created and the routers.
  
  The error I receved was:
  
  Error: Unable to retrieve usage information.
  Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-538dfec9-8464-4a92-9b03-2c610c3eff07)
+ 
+ Environment
+ ================================================
+ Openstack Ussuri Release
  
  The log file
  
  =============================================
  
  2021-02-09 11:43:02.838 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:02.843 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:02.848 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:03.887 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] [cad79092-0027-438d-9cce-8fbe181f4ff3] Reconnected to AMQP server on 172.25.8.252:5672 via [amqp] client with port 53754.
  2021-02-09 11:43:04.573 1506671 INFO nova.api.openstack.requestlog [req-d2ca1be8-7bb3-45d1-91e2-0bf598112be7 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08/servers/detail?project_id=efc6ffe3908d447987ba5ed6d798ae08" status: 200 len: 17406 microversion: 2.42 time: 1.737148
  2021-02-09 11:43:05.501 1506667 INFO nova.api.openstack.requestlog [req-02697d51-58f8-4528-afe2-2d24d8e802a5 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08" status: 404 len: 112 microversion: - time: 0.186829
  2021-02-09 11:43:05.510 1506670 INFO nova.api.openstack.requestlog [req-2fd6b62e-cb1f-4919-88d2-d41af9b3a7b9 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/" status: 200 len: 386 microversion: 2.1 time: 0.006624
  2021-02-09 11:43:06.220 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:06.231 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:06.235 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:06.737 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] [67d970a3-15cd-4b87-8b2d-a47bd906b100] Reconnected to AMQP server on 172.25.8.252:5672 via [amqp] client with port 53794.
  2021-02-09 11:43:08.143 1506666 INFO nova.api.openstack.requestlog [req-11e3f941-ecee-45e7-9964-d70b063a0290 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08/servers/detail?project_id=efc6ffe3908d447987ba5ed6d798ae08" status: 200 len: 17406 microversion: 2.42 time: 2.034052
  2021-02-09 11:48:43.174 1506669 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:48:43.178 1506669 ERROR oslo.messaging._drivers.impl_rabbit [-] [7916d861-ee6d-484c-9cab-1cd4946ec11d] AMQP server on 172.25.8.252:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer
  2021-02-09 11:48:43.388 1506669 INFO nova.api.openstack.requestlog [req-8b40bd1f-371b-40a2-aa78-6f073b80342a 12d3b5fee3be4e9c94bb60c8acc14997 781dc171cf7642dfa11c6649989fcd2d - default default] 172.25.8.252 "GET /v2.1/781dc171cf7642dfa11c6649989fcd2d/servers/detail?all_tenants=True&changes-since=2021-02-09T06%3A08%3A43.495476%2B00%3A00" status: 200 len: 15 microversion: 2.1 time: 0.216355
  2021-02-09 14:34:31.067 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 14:34:31.119 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 14:34:31.124 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 14:34:31.130 1506667 ERROR oslo.messaging._drivers.impl_rabbit [-] [eea1f819-0d22-4034-bbfa-83b93a9be40e] AMQP server on 172.25.8.252:5672 is unreachable: [Errno 104

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1915143

Title:
  Unexpected API Error. Please report this at
  http://bugs.launchpad.net/nova/ and attach the Nova API log if
  possible. <class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID:
  req-538dfec9-8464-4a92-9b03-2c610c3eff07)

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  =================================================

  I am logged in as admin and created some instances and images as well
  , they were visible for one week but suddenly when I tried to check
  today I wasnt able to see any images or instances or volumes , however
  i was able to see the networks I created and the routers.

  The error I receved was:

  Error: Unable to retrieve usage information.
  Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-538dfec9-8464-4a92-9b03-2c610c3eff07)

  Environment
  ================================================
  Openstack Ussuri Release

  The log file

  =============================================

  2021-02-09 11:43:02.838 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:02.843 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:02.848 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:03.887 1506671 INFO oslo.messaging._drivers.impl_rabbit [-] [cad79092-0027-438d-9cce-8fbe181f4ff3] Reconnected to AMQP server on 172.25.8.252:5672 via [amqp] client with port 53754.
  2021-02-09 11:43:04.573 1506671 INFO nova.api.openstack.requestlog [req-d2ca1be8-7bb3-45d1-91e2-0bf598112be7 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08/servers/detail?project_id=efc6ffe3908d447987ba5ed6d798ae08" status: 200 len: 17406 microversion: 2.42 time: 1.737148
  2021-02-09 11:43:05.501 1506667 INFO nova.api.openstack.requestlog [req-02697d51-58f8-4528-afe2-2d24d8e802a5 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08" status: 404 len: 112 microversion: - time: 0.186829
  2021-02-09 11:43:05.510 1506670 INFO nova.api.openstack.requestlog [req-2fd6b62e-cb1f-4919-88d2-d41af9b3a7b9 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/" status: 200 len: 386 microversion: 2.1 time: 0.006624
  2021-02-09 11:43:06.220 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:06.231 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:06.235 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:43:06.737 1506666 INFO oslo.messaging._drivers.impl_rabbit [-] [67d970a3-15cd-4b87-8b2d-a47bd906b100] Reconnected to AMQP server on 172.25.8.252:5672 via [amqp] client with port 53794.
  2021-02-09 11:43:08.143 1506666 INFO nova.api.openstack.requestlog [req-11e3f941-ecee-45e7-9964-d70b063a0290 5102000f220147a197c16910458eafb3 efc6ffe3908d447987ba5ed6d798ae08 - default default] 172.25.8.252 "GET /v2.1/efc6ffe3908d447987ba5ed6d798ae08/servers/detail?project_id=efc6ffe3908d447987ba5ed6d798ae08" status: 200 len: 17406 microversion: 2.42 time: 2.034052
  2021-02-09 11:48:43.174 1506669 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 11:48:43.178 1506669 ERROR oslo.messaging._drivers.impl_rabbit [-] [7916d861-ee6d-484c-9cab-1cd4946ec11d] AMQP server on 172.25.8.252:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer
  2021-02-09 11:48:43.388 1506669 INFO nova.api.openstack.requestlog [req-8b40bd1f-371b-40a2-aa78-6f073b80342a 12d3b5fee3be4e9c94bb60c8acc14997 781dc171cf7642dfa11c6649989fcd2d - default default] 172.25.8.252 "GET /v2.1/781dc171cf7642dfa11c6649989fcd2d/servers/detail?all_tenants=True&changes-since=2021-02-09T06%3A08%3A43.495476%2B00%3A00" status: 200 len: 15 microversion: 2.1 time: 0.216355
  2021-02-09 14:34:31.067 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 14:34:31.119 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 14:34:31.124 1506667 INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection
  2021-02-09 14:34:31.130 1506667 ERROR oslo.messaging._drivers.impl_rabbit [-] [eea1f819-0d22-4034-bbfa-83b93a9be40e] AMQP server on 172.25.8.252:5672 is unreachable: [Errno 104

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1915143/+subscriptions
Follow ups

[Bug 1915143] Re: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'oslo_db.exception.DBError'> (HTTP 500) (Request-ID: req-538dfec9-8464-4a92-9b03-2c610c3eff07)
From: Balazs Gibizer, 2021-02-11