yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #84941
[Bug 1912343] [NEW] Nova returns 'The requested availability zone is not available (HTTP 400)' on any cell errors
Public bug reported:
Description
===========
In case of for example DB errors `openstack server create --availability-zone ...` command returns
"The requested availability zone is not available (HTTP 400)" instead of actual 50x error.
This considered as client error and e.g. stops Heat to retry requests.
For example:
root@dev01:~# openstack server create --image cirros-0.5.1-x86_64-disk --flavor c1 vm5 --net public --availability-zone nova
The requested availability zone is not available (HTTP 400) (Request-ID: req-50190228-51ac-4303-ad3e-d8b920bb7ad8)
Logs:
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(*args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: DEBUG nova.objects.service [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] ! service get_all {{(pid=4872) get_all /opt/stack/nova/nova/objects/service.py:632}}
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 558d4e82-cdc0-4020-a56b-6835326f58ec: ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(*args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 00000000-0000-0000-0000-000000000000 is not responding and hence skipped from the results.
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 558d4e82-cdc0-4020-a56b-6835326f58ec is not responding and hence skipped from the results.
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: INFO nova.api.openstack.wsgi [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] HTTP exception thrown: The requested availability zone is not available
Under the same conditions, `openstack compute service list` returns an
empty success result.
Steps to reproduce
==================
To emulate runtime errors something like that can be used:
@@ -629,6 +629,10 @@ class ServiceList(base.ObjectListBase,
base.NovaObject):
@base.remotable_classmethod
def get_all(cls, context, disabled=None, set_zones=False):
+ import datetime
+ if datetime.datetime.now() > datetime.datetime(2021, 1, 19, 12, 7, 0, 976979):
+ raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
db_services = db.service_get_all(context, disabled=disabled)
if set_zones:
db_services = availability_zones.set_availability_zones(
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1912343
Title:
Nova returns 'The requested availability zone is not available (HTTP
400)' on any cell errors
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
In case of for example DB errors `openstack server create --availability-zone ...` command returns
"The requested availability zone is not available (HTTP 400)" instead of actual 50x error.
This considered as client error and e.g. stops Heat to retry requests.
For example:
root@dev01:~# openstack server create --image cirros-0.5.1-x86_64-disk --flavor c1 vm5 --net public --availability-zone nova
The requested availability zone is not available (HTTP 400) (Request-ID: req-50190228-51ac-4303-ad3e-d8b920bb7ad8)
Logs:
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(*args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: DEBUG nova.objects.service [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] ! service get_all {{(pid=4872) get_all /opt/stack/nova/nova/objects/service.py:632}}
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Error gathering result from cell 558d4e82-cdc0-4020-a56b-6835326f58ec: ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context Traceback (most recent call last):
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/context.py", line 426, in gather_result
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(*args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/usr/local/lib/python3.6/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context result = fn(cls, context, *args, **kwargs)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context File "/opt/stack/nova/nova/objects/service.py", line 635, in get_all
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context ValueError: Cannot get service list (artificial error to help reproduce a bug)
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: ERROR nova.context
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 00000000-0000-0000-0000-000000000000 is not responding and hence skipped from the results.
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: WARNING nova.compute.api [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] Cell 558d4e82-cdc0-4020-a56b-6835326f58ec is not responding and hence skipped from the results.
Jan 19 12:37:32 dev01 devstack@n-api.service[4869]: INFO nova.api.openstack.wsgi [None req-50190228-51ac-4303-ad3e-d8b920bb7ad8 admin admin] HTTP exception thrown: The requested availability zone is not available
Under the same conditions, `openstack compute service list` returns an
empty success result.
Steps to reproduce
==================
To emulate runtime errors something like that can be used:
@@ -629,6 +629,10 @@ class ServiceList(base.ObjectListBase,
base.NovaObject):
@base.remotable_classmethod
def get_all(cls, context, disabled=None, set_zones=False):
+ import datetime
+ if datetime.datetime.now() > datetime.datetime(2021, 1, 19, 12, 7, 0, 976979):
+ raise ValueError('Cannot get service list (artificial error to help reproduce a bug)')
db_services = db.service_get_all(context, disabled=disabled)
if set_zones:
db_services = availability_zones.set_availability_zones(
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1912343/+subscriptions