yahoo-eng-team team mailing list archive

[Ihar Hrachyshka <2094846@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

This happened in gate:

https://763b7c31d44017973024-0677f910aec9b59957afc606ff2e533f.ssl.cf2.rackcdn.com/938106/3/check/neutron-
functional/532a7a3/testr_results.html

ft1.1: neutron.tests.functional.services.trunk.drivers.ovn.test_trunk_driver.TestOVNTrunkDriver.test_subport_deletetesttools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/services/trunk/drivers/ovn/test_trunk_driver.py", line 137, in test_subport_delete
    self.assertTrue(lsp_event.wait())
  File "/usr/lib/python3.12/unittest/case.py", line 727, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true

The error is because LSP update event was not received by the ovsdb-
monitor. The reason why it did not is because OVN db transaction that
was supposed to update the SB port-binding (to remove subport) failed
with:

2025-01-13 17:07:36.572 304173 DEBUG neutron.services.trunk.drivers.ovn.trunk_driver [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Unsetting parent for subport 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 _unset_binding_profile /home/zuul/src/opendev.org/openstack/neutron/neutron/services/trunk/drivers/ovn/trunk_driver.py:118
2025-01-13 17:07:36.617 304173 DEBUG neutron.services.trunk.drivers.ovn.trunk_driver [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Done unsetting parent for subport 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 _unset_binding_profile /home/zuul/src/opendev.org/openstack/neutron/neutron/services/trunk/drivers/ovn/trunk_driver.py:155
2025-01-13 17:07:36.621 304173 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Traceback (most recent call last):
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run
    txn.results.put(txn.do_commit())
                    ^^^^^^^^^^^^^^^
  File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
    command.run_idl(txn)
  File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 956, in run_idl
    raise ovn_exc.RevisionConflict(
neutron.common.ovn.exceptions.RevisionConflict: OVN revision number for 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 (type: ports) is higher than the given resource. Skipping update

2025-01-13 17:07:36.622 304173 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Transaction aborted. Reason: OVN revision number for 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 (type: ports) is higher than the given resource. Skipping update
2025-01-13 17:07:36.639 304173 DEBUG neutron.db.ovn_revision_numbers_db [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Skip bumping the revision number for 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 (type: ports) to 1. A higher version is already registered in the database (2) bump_revision /home/zuul/src/opendev.org/openstack/neutron/neutron/db/ovn_revision_numbers_db.py:194

The OVN revision number == 2 while trunk plugin attempts to update with
revision number == 1.

---

I don't know why trunk plugin believes the neutron-db record for the
subport still has revision number == 1. The logic of the test case
suggests it should be 2.

The test goes as follows:

1. create a (sub)port. (It is created with rev=1)
2. attach the port to trunk. (This bumps to 2.)
3. detach from trunk. At this point I would expect the db object to have rev-number>=2, but it's one, as per the log message quoted above.

It's as if the neutron db layer did not persist previous revision bump.

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: db functional-tests gate-failure ovn

** Tags added: gate

** Tags removed: gate
** Tags added: db functional-tests gate-failure ovn

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2094846

Title:
  neutron.tests.functional.services.trunk.drivers.ovn.test_trunk_driver.TestOVNTrunkDriver.test_subport_delete
  fails because remove_subports fails to update OVN db due to revision
  number inconsistency

Status in neutron:
  New

Bug description:
  This happened in gate:

  https://763b7c31d44017973024-0677f910aec9b59957afc606ff2e533f.ssl.cf2.rackcdn.com/938106/3/check/neutron-
  functional/532a7a3/testr_results.html

  ft1.1: neutron.tests.functional.services.trunk.drivers.ovn.test_trunk_driver.TestOVNTrunkDriver.test_subport_deletetesttools.testresult.real._StringException: Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 178, in func
      return f(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/functional/services/trunk/drivers/ovn/test_trunk_driver.py", line 137, in test_subport_delete
      self.assertTrue(lsp_event.wait())
    File "/usr/lib/python3.12/unittest/case.py", line 727, in assertTrue
      raise self.failureException(msg)
  AssertionError: False is not true

  The error is because LSP update event was not received by the ovsdb-
  monitor. The reason why it did not is because OVN db transaction that
  was supposed to update the SB port-binding (to remove subport) failed
  with:

  2025-01-13 17:07:36.572 304173 DEBUG neutron.services.trunk.drivers.ovn.trunk_driver [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Unsetting parent for subport 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 _unset_binding_profile /home/zuul/src/opendev.org/openstack/neutron/neutron/services/trunk/drivers/ovn/trunk_driver.py:118
  2025-01-13 17:07:36.617 304173 DEBUG neutron.services.trunk.drivers.ovn.trunk_driver [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Done unsetting parent for subport 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 _unset_binding_profile /home/zuul/src/opendev.org/openstack/neutron/neutron/services/trunk/drivers/ovn/trunk_driver.py:155
  2025-01-13 17:07:36.621 304173 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Traceback (most recent call last):
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run
      txn.results.put(txn.do_commit())
                      ^^^^^^^^^^^^^^^
    File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
      command.run_idl(txn)
    File "/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 956, in run_idl
      raise ovn_exc.RevisionConflict(
  neutron.common.ovn.exceptions.RevisionConflict: OVN revision number for 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 (type: ports) is higher than the given resource. Skipping update

  2025-01-13 17:07:36.622 304173 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Transaction aborted. Reason: OVN revision number for 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 (type: ports) is higher than the given resource. Skipping update
  2025-01-13 17:07:36.639 304173 DEBUG neutron.db.ovn_revision_numbers_db [None req-548c0895-271f-4af7-87ab-178e41d83359 - 46f70361-ba71-4bd0-9769-3573fd227c4b - - - -] Skip bumping the revision number for 3787fbf2-d08d-4c33-bd71-8065b2b0d3d3 (type: ports) to 1. A higher version is already registered in the database (2) bump_revision /home/zuul/src/opendev.org/openstack/neutron/neutron/db/ovn_revision_numbers_db.py:194

  The OVN revision number == 2 while trunk plugin attempts to update
  with revision number == 1.

  ---

  I don't know why trunk plugin believes the neutron-db record for the
  subport still has revision number == 1. The logic of the test case
  suggests it should be 2.

  The test goes as follows:

  1. create a (sub)port. (It is created with rev=1)
  2. attach the port to trunk. (This bumps to 2.)
  3. detach from trunk. At this point I would expect the db object to have rev-number>=2, but it's one, as per the log message quoted above.

  It's as if the neutron db layer did not persist previous revision
  bump.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2094846/+subscriptions