yahoo-eng-team team mailing list archive

[OpenStack Infra <1959677@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/nova/+/831084
Committed: https://opendev.org/openstack/nova/commit/1c8122a25f50b40934af127d7717b55794ff38b5
Submitter: "Zuul (22348)"
Branch:    master

commit 1c8122a25f50b40934af127d7717b55794ff38b5
Author: melanie witt <melwittt@xxxxxxxxx>
Date:   Sat Feb 26 19:51:18 2022 +0000

    Retry in CellDatabases fixture when global DB state changes
    
    There is a NOTE in the CellDatabases code about an unlikely but
    possible race that can occur between taking the writer lock to set
    the last DB context manager and taking the reader lock to call
    target_cell(). When the race is detected, a RuntimeError is raised.
    
    We can handle the race by retrying setting the last DB context manager
    when the race is detected, as described in the NOTE.
    
    Closes-Bug: #1959677
    
    Change-Id: I5c0607ce5910dce581ab9360cc7fc69ba9673f35


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1959677

Title:
  nova functional tests fail with: "Global DB state changed underneath
  us"

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/nova/nova/api/openstack/wsgi.py", line 658, in wrapped
      return f(*args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/nova/nova/api/validation/__init__.py", line 110, in wrapper
      return func(*args, **kwargs)
    File "/home/zuul/src/opendev.org/openstack/nova/nova/api/openstack/compute/services.py", line 439, in update
      service = self.host_api.service_get_by_id(context, service_id)
    File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/api.py", line 5982, in service_get_by_id
      return _find_service_in_cell(context, service_id=service_id)
    File "/home/zuul/src/opendev.org/openstack/nova/nova/compute/api.py", line 5824, in _find_service_in_cell
      with nova_context.target_cell(context, cell) as cctxt:
    File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
      return next(self.gen)
    File "/home/zuul/src/opendev.org/openstack/nova/nova/tests/fixtures/nova.py", line 488, in _wrap_target_cell
      raise RuntimeError('Global DB state changed underneath us')
  RuntimeError: Global DB state changed underneath us

  Example failure:
  https://zuul.opendev.org/t/openstack/build/81e8fc29d2a44e1b93ecea30016516bd/logs

  The frequency of the issue increased recently to happening daily:
  https://paste.opendev.org/show/812466/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1959677/+subscriptions