← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #84941

[Bug 1912343] [NEW] Nova returns 'The requested availability zone is not available (HTTP 400)' on any cell errors

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

Description
===========

In case of for example DB errors `openstack server create --availability-zone ...` command returns
"The requested availability zone is not available (HTTP 400)" instead of actual 50x error.
This considered as client error and e.g. stops Heat to retry requests.

For example:

root@dev01:~# openstack server create --image cirros-0.5.1-x86_64-disk --flavor c1 vm5 --net public --availability-zone nova
The requested availability zone is not available (HTTP 400) (Request-ID: req-50190228-51ac-4303-ad3e-d8b920bb7ad8)

Logs:
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/context.py", line 426, in gather_result
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(*args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(cls, context, *args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: DEBUG nova.objects.service [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] ! service get_all {{(pid=4872) get_all /opt/stack/nova/nova/objects/service.py:632}}
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 558d4e82-cdc0-4020-a56b-6835326f58ec: ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/context.py", line 426, in gather_result
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(*args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(cls, context, *args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 00000000-0000-0000-0000-000000000000 is not responding and hence skipped from the results.
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 558d4e82-cdc0-4020-a56b-6835326f58ec is not responding and hence skipped from the results.
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: INFO nova.api.openstack.wsgi [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] HTTP exception thrown: The requested availability zone is not available

Under the same conditions, `openstack compute service list` returns an
empty success result.

Steps to reproduce
==================

To emulate runtime errors something like that can be used:

@@ -629,6 +629,10 @@ class ServiceList(base.ObjectListBase,
base.NovaObject):

     @base.remotable_classmethod
     def get_all(cls, context, disabled=None, set_zones=False):
+        import datetime
+        if datetime.datetime.now() > datetime.datetime(2021, 1, 19, 12, 7, 0, 976979):
+            raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
         db_services = db.service_get_all(context, disabled=disabled)
         if set_zones:
             db_services = availability_zones.set_availability_zones(

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1912343

Title:
  Nova returns 'The requested availability zone is not available (HTTP
  400)' on any cell errors

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  ===========

  In case of for example DB errors `openstack server create --availability-zone ...` command returns
  "The requested availability zone is not available (HTTP 400)" instead of actual 50x error.
  This considered as client error and e.g. stops Heat to retry requests.

  For example:

  root@dev01:~# openstack server create --image cirros-0.5.1-x86_64-disk --flavor c1 vm5 --net public --availability-zone nova
  The requested availability zone is not available (HTTP 400) (Request-ID: req-50190228-51ac-4303-ad3e-d8b920bb7ad8)

  Logs:
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ValueError: Cannot get service list (artificial error to help reproduce a bug)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/context.py", line 426, in gather_result
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(*args, **kwargs)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(cls, context, *args, **kwargs)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: DEBUG nova.objects.service [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] ! service get_all {{(pid=4872) get_all /opt/stack/nova/nova/objects/service.py:632}}
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 558d4e82-cdc0-4020-a56b-6835326f58ec: ValueError: Cannot get service list (artificial error to help reproduce a bug)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/context.py", line 426, in gather_result
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(*args, **kwargs)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     result = fn(cls, context, *args, **kwargs)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context   File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context     raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 00000000-0000-0000-0000-000000000000 is not responding and hence skipped from the results.
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 558d4e82-cdc0-4020-a56b-6835326f58ec is not responding and hence skipped from the results.
  Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: INFO nova.api.openstack.wsgi [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] HTTP exception thrown: The requested availability zone is not available

  Under the same conditions, `openstack compute service list` returns an
  empty success result.

  Steps to reproduce
  ==================

  To emulate runtime errors something like that can be used:

  @@ -629,6 +629,10 @@ class ServiceList(base.ObjectListBase,
  base.NovaObject):

       @base.remotable_classmethod
       def get_all(cls, context, disabled=None, set_zones=False):
  +        import datetime
  +        if datetime.datetime.now() > datetime.datetime(2021, 1, 19, 12, 7, 0, 976979):
  +            raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
           db_services = db.service_get_all(context, disabled=disabled)
           if set_zones:
               db_services = availability_zones.set_availability_zones(

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1912343/+subscriptions
  Thread Previous • Date Previous • • Thread Next