yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2037500] [NEW] OVSDB transaction returned TRY_AGAIN, retrying do_commit

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Attasit <2037500@xxxxxxxxxxxxxxxxxx>
Date: Wed, 27 Sep 2023 05:26:56 -0000
Reply-to: Bug 2037500 <2037500@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Trying to create instance and got error when it's trying to attach the
port to instance about details below on neutron server.

2023-09-27 09:58:10.725 716 DEBUG ovsdbapp.backend.ovs_idl.transaction [req-2df7a23e-8b9f-409c-a35e-9b78edb6bce1 - - - - -] OVSDB transaction returned TRY_AGAIN, retrying do_commit /usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:97
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-ca61167b-aca9-46a2-81fb-8f8e3ebba349 - - - - -] OVS database connection to OVN_Northbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: Exception: Timeout
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 68, in start_connection
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.ovsdb_connection.start()
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 79, in start
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     idlutils.wait_for_change(self.idl, self.timeout)
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 219, in wait_for_change
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     raise Exception("Timeout")  # TODO(twilson) use TimeoutException?
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Exception: Timeout
2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn

and later the error still come below.


2023-09-27 12:07:36.849 747 ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
2023-09-27 12:07:36.849 747 ERROR ovsdbapp.backend.ovs_idl.transaction [req-7f9163da-8faf-4509-b650-aedfdf4ff303 - - - - -] Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 122, in run
    txn.results.put(txn.do_commit())
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 119, in do_commit
    raise RuntimeError(msg)
RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it

2023-09-27 12:07:36.849 747 ERROR futurist.periodics [req-7f9163da-8faf-4509-b650-aedfdf4ff303 - - - - -] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_ha_chassis_group_address' (it runs every 600.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
2023-09-27 12:07:36.849 747 ERROR futurist.periodics Traceback (most recent call last):
2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/futurist/periodics.py", line 293, in run
2023-09-27 12:07:36.849 747 ERROR futurist.periodics     work()
2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/futurist/periodics.py", line 67, in __call__
2023-09-27 12:07:36.849 747 ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/futurist/periodics.py", line 181, in decorator
2023-09-27 12:07:36.849 747 ERROR futurist.periodics     return f(*args, **kwargs)
2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 622, in check_for_ha_chassis_group_address
2023-09-27 12:07:36.849 747 ERROR futurist.periodics     priority -= 1
2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2023-09-27 12:07:36.849 747 ERROR futurist.periodics     next(self.gen)
2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 185, in transaction

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2037500

Title:
  OVSDB transaction returned TRY_AGAIN, retrying do_commit

Status in neutron:
  New

Bug description:
  Trying to create instance and got error when it's trying to attach the
  port to instance about details below on neutron server.

  2023-09-27 09:58:10.725 716 DEBUG ovsdbapp.backend.ovs_idl.transaction [req-2df7a23e-8b9f-409c-a35e-9b78edb6bce1 - - - - -] OVSDB transaction returned TRY_AGAIN, retrying do_commit /usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py:97
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [req-ca61167b-aca9-46a2-81fb-8f8e3ebba349 - - - - -] OVS database connection to OVN_Northbound failed with error: 'Timeout'. Verify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: Exception: Timeout
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 68, in start_connection
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.ovsdb_connection.start()
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 79, in start
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     idlutils.wait_for_change(self.idl, self.timeout)
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 219, in wait_for_change
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     raise Exception("Timeout")  # TODO(twilson) use TimeoutException?
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Exception: Timeout
  2023-09-27 09:58:10.724 723 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn

  and later the error still come below.

  
  2023-09-27 12:07:36.849 747 ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
  2023-09-27 12:07:36.849 747 ERROR ovsdbapp.backend.ovs_idl.transaction [req-7f9163da-8faf-4509-b650-aedfdf4ff303 - - - - -] Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 122, in run
      txn.results.put(txn.do_commit())
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 119, in do_commit
      raise RuntimeError(msg)
  RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it

  2023-09-27 12:07:36.849 747 ERROR futurist.periodics [req-7f9163da-8faf-4509-b650-aedfdf4ff303 - - - - -] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_ha_chassis_group_address' (it runs every 600.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics Traceback (most recent call last):
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/futurist/periodics.py", line 293, in run
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics     work()
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/futurist/periodics.py", line 67, in __call__
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/futurist/periodics.py", line 181, in decorator
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics     return f(*args, **kwargs)
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 622, in check_for_ha_chassis_group_address
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics     priority -= 1
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics     next(self.gen)
  2023-09-27 12:07:36.849 747 ERROR futurist.periodics   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 185, in transaction

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2037500/+subscriptions
Follow ups

[Bug 2037500] Re: OVSDB transaction returned TRY_AGAIN, retrying do_commit
From: Brian Haley, 2023-10-03