yahoo-eng-team team mailing list archive

[Elod Illes <1901120@xxxxxxxxxxxxxxxxxx>]

** Changed in: nova/ussuri
       Status: Fix Committed => Fix Released

** Changed in: nova/victoria
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1901120

Title:
  ResourceProviderUpdateFailed when total available resource is 0

Status in OpenStack Compute (nova):
  Fix Released
Status in OpenStack Compute (nova) ussuri series:
  Fix Released
Status in OpenStack Compute (nova) victoria series:
  Fix Released

Bug description:
  This got reported downstream [1] where if a compute node has no
  available (zero) inventory for a resource class, the libvirt driver
  attempts to PUT the 'total' inventory of the resource as 0 and the
  placement API does not allow it. To represent a 'total' of 0 in
  placement for a resource, one must omit that resource from the PUT
  request instead of trying to set it to zero.

  The following traceback is emitted when an attempt to update 'total'
  inventory to 0 occurs:

  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager nova.exception.ResourceProviderUpdateFailed: Failed to update resource provider via URL /resource_providers/5812a7ab-f8e7-4c57-aba8-f283fedfd04b/inventories: {"errors": [{"status": 400, "title": "Bad Request", "detail": "The server could not comply with the request since it is either malformed or otherwise incorrect.\n\n JSON does not validate: 0 is less than the minimum of 1  Failed validating 'minimum' in schema['properties']['inventories']['patternProperties']['^[A-Z0-9_]+$']['properties']['total']:     {'maximum': 2147483647, 'minimum': 1, 'type': 'integer'}  On instance['inventories']['DISK_GB']['total']:     0  ", "code": "placement.undefined_code", "request_id": "req-7fc88053-8e43-4c18-9810-73ad05c3a993"}]}
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager 
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager During handling of the above exception, another exception occurred:
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager 
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager Traceback (most recent call last):
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 8740, in _update_available_resource_for_node
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     startup=startup)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 887, in update_available_resource
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     self._update_available_resource(context, resources, startup=startup)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/oslo_concurrency/lockutils.py", line 328, in inner
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     return f(*args, **kwargs)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 972, in _update_available_resource
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     self._update(context, cn, startup=startup)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1237, in _update
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     self._update_to_placement(context, compute_node, startup)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/retrying.py", line 68, in wrapped_f
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     return Retrying(*dargs, **dkw).call(f, *args, **kw)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/retrying.py", line 223, in call
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     return attempt.get(self._wrap_exception)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/retrying.py", line 261, in get
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     six.reraise(self.value[0], self.value[1], self.value[2])
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     raise value
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/retrying.py", line 217, in call
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/compute/resource_tracker.py", line 1215, in _update_to_placement
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     allocations=allocs)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 1421, in update_from_provider_tree
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     self.set_traits_for_provider(context, pd.uuid, pd.traits)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib64/python3.6/contextlib.py", line 99, in __exit__
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     self.gen.throw(type, value, traceback)
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager   File "/usr/lib/python3.6/site-packages/nova/scheduler/client/report.py", line 1341, in catch_all
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager     raise exception.ResourceProviderSyncFailed()
  2020-10-20 11:32:45.484 7 ERROR nova.compute.manager nova.exception.ResourceProviderSyncFailed: Failed to synchronize the placement service with resource provider information supplied by the compute host.

  [1] https://bugzilla.redhat.com/show_bug.cgi?id=1889723

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1901120/+subscriptions