yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2000495] [NEW] See error "HashRingIsEmpty" when create instance

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Gavin Guan <2000495@xxxxxxxxxxxxxxxxxx>
Date: Tue, 27 Dec 2022 03:14:03 -0000
Reply-to: Bug 2000495 <2000495@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

neutron show below errors when create an instance

2022-12-27 10:31:07.227 1205638 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205637 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205630 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-17d58c66-64ec-47d9-8cca-4afb0d6e6e83 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205632 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-586ce94d-e22c-4b71-b25c-ceb44921d4b0 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205634 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205639 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205633 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-3f48b6b3-1b77-4dce-a78d-cdaa0c9e0b57 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.230 1205636 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-e4d88a33-459d-4933-b59d-86cafbbf6fe8 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205638 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205639 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205636 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-4e648561-04c9-4afa-b117-226461807d14 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205637 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205633 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-71a3cffe-7e68-44f7-b31d-aa0d5f8b09f1 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205632 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-add99db6-d820-4a04-9fe6-b08b57884c85 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205630 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205634 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205635 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-bd75e02f-a59f-4193-91f5-7cf7147219ea - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.231 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.232 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-46908903-6ee7-4a7d-82da-3934a51e1190 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.232 1205635 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:07.240 1205641 INFO neutron.db.ovn_revision_numbers_db [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Successfully bumped revision number for resource 54e9a52c-b0f2-4a48-98df-3875fbdfbc89 (type: ports) to 3
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.transaction [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
    txn.results.put(txn.do_commit())
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
    command.run_idl(txn)
  File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
    entity = self.api.lookup(self.lookup_table, self.entity)
  File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
    return super().lookup(table, record, default=default, timeout=timeout,
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
    return self._lookup(table, record)
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
    row = idlutils.row_by_value(self, rl.table, rl.column, record)
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
    raise RowNotFound(table=table, col=column, match=match)
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab

2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Error executing command (PgAclAddCommand): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     t.add(self)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     yield t
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     del self._nested_txns_map[cur_thread_id]
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     command.run_idl(txn)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     entity = self.api.lookup(self.lookup_table, self.entity)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return super().lookup(table, record, default=default, timeout=timeout,
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return self._lookup(table, record)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     row = idlutils.row_by_value(self, rl.table, rl.column, record)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise RowNotFound(table=table, col=column, match=match)
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command 
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Maintenance task: Failed to fix resource 1628ebd0-4f0e-4541-b607-0611168ed4c3 (type: security_group_rules): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 392, in check_for_inconsistencies
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 231, in _fix_create_update
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_create'](context, n_obj)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2269, in create_security_group_rule
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._process_security_group_rule(rule)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2262, in _process_security_group_rule
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_acl.update_acls_for_security_group(
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/common/ovn/acl.py", line 305, in update_acls_for_security_group
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn.pg_acl_add(**acl, may_exist=True).execute(check_error=True)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     t.add(self)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     yield t
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     del self._nested_txns_map[cur_thread_id]
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     command.run_idl(txn)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     entity = self.api.lookup(self.lookup_table, self.entity)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return super().lookup(table, record, default=default, timeout=timeout,
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return self._lookup(table, record)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     row = idlutils.row_by_value(self, rl.table, rl.column, record)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RowNotFound(table=table, col=column, match=match)
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.transaction [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
    txn.results.put(txn.do_commit())
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
    command.run_idl(txn)
  File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
    entity = self.api.lookup(self.lookup_table, self.entity)
  File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
    return super().lookup(table, record, default=default, timeout=timeout,
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
    return self._lookup(table, record)
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
    row = idlutils.row_by_value(self, rl.table, rl.column, record)
  File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
    raise RowNotFound(table=table, col=column, match=match)
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab

2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Error executing command (PgAclAddCommand): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     t.add(self)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     yield t
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     del self._nested_txns_map[cur_thread_id]
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     command.run_idl(txn)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     entity = self.api.lookup(self.lookup_table, self.entity)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return super().lookup(table, record, default=default, timeout=timeout,
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return self._lookup(table, record)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     row = idlutils.row_by_value(self, rl.table, rl.column, record)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise RowNotFound(table=table, col=column, match=match)
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command 
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Maintenance task: Failed to fix resource e7813967-9b79-4384-b355-bff5c0de88f1 (type: security_group_rules): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 392, in check_for_inconsistencies
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 231, in _fix_create_update
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_create'](context, n_obj)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2269, in create_security_group_rule
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._process_security_group_rule(rule)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2262, in _process_security_group_rule
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_acl.update_acls_for_security_group(
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/common/ovn/acl.py", line 305, in update_acls_for_security_group
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn.pg_acl_add(**acl, may_exist=True).execute(check_error=True)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     t.add(self)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     yield t
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     del self._nested_txns_map[cur_thread_id]
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     command.run_idl(txn)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     entity = self.api.lookup(self.lookup_table, self.entity)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return super().lookup(table, record, default=default, timeout=timeout,
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return self._lookup(table, record)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     row = idlutils.row_by_value(self, rl.table, rl.column, record)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RowNotFound(table=table, col=column, match=match)
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
2022-12-27 10:31:07.268 1205641 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Maintenance task: Synchronization finished (took 0.07 seconds)
2022-12-27 10:31:09.358 1205633 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-71a3cffe-7e68-44f7-b31d-aa0d5f8b09f1 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:09.359 1205632 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-add99db6-d820-4a04-9fe6-b08b57884c85 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:09.359 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-46908903-6ee7-4a7d-82da-3934a51e1190 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:09.363 1205630 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:09.367 1205635 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
2022-12-27 10:31:09.375 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2000495

Title:
  See error "HashRingIsEmpty" when create instance

Status in neutron:
  New

Bug description:
  neutron show below errors when create an instance

  2022-12-27 10:31:07.227 1205638 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205637 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205630 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-17d58c66-64ec-47d9-8cca-4afb0d6e6e83 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205632 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-586ce94d-e22c-4b71-b25c-ceb44921d4b0 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205634 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205639 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205633 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-3f48b6b3-1b77-4dce-a78d-cdaa0c9e0b57 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.230 1205636 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-e4d88a33-459d-4933-b59d-86cafbbf6fe8 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205638 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205639 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205636 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-4e648561-04c9-4afa-b117-226461807d14 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205637 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205633 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-71a3cffe-7e68-44f7-b31d-aa0d5f8b09f1 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205632 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-add99db6-d820-4a04-9fe6-b08b57884c85 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205630 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205634 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205635 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-bd75e02f-a59f-4193-91f5-7cf7147219ea - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.231 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.232 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-46908903-6ee7-4a7d-82da-3934a51e1190 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.232 1205635 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:07.240 1205641 INFO neutron.db.ovn_revision_numbers_db [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Successfully bumped revision number for resource 54e9a52c-b0f2-4a48-98df-3875fbdfbc89 (type: ports) to 3
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.transaction [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
      txn.results.put(txn.do_commit())
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
      command.run_idl(txn)
    File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
      entity = self.api.lookup(self.lookup_table, self.entity)
    File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
      return super().lookup(table, record, default=default, timeout=timeout,
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
      return self._lookup(table, record)
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
      row = idlutils.row_by_value(self, rl.table, rl.column, record)
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
      raise RowNotFound(table=table, col=column, match=match)
  ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab

  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Error executing command (PgAclAddCommand): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     t.add(self)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     yield t
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     del self._nested_txns_map[cur_thread_id]
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     command.run_idl(txn)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     entity = self.api.lookup(self.lookup_table, self.entity)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return super().lookup(table, record, default=default, timeout=timeout,
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return self._lookup(table, record)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     row = idlutils.row_by_value(self, rl.table, rl.column, record)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise RowNotFound(table=table, col=column, match=match)
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.254 1205641 ERROR ovsdbapp.backend.ovs_idl.command 
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Maintenance task: Failed to fix resource 1628ebd0-4f0e-4541-b607-0611168ed4c3 (type: security_group_rules): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 392, in check_for_inconsistencies
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 231, in _fix_create_update
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_create'](context, n_obj)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2269, in create_security_group_rule
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._process_security_group_rule(rule)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2262, in _process_security_group_rule
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_acl.update_acls_for_security_group(
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/common/ovn/acl.py", line 305, in update_acls_for_security_group
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn.pg_acl_add(**acl, may_exist=True).execute(check_error=True)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     t.add(self)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     yield t
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     del self._nested_txns_map[cur_thread_id]
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     command.run_idl(txn)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     entity = self.api.lookup(self.lookup_table, self.entity)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return super().lookup(table, record, default=default, timeout=timeout,
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return self._lookup(table, record)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     row = idlutils.row_by_value(self, rl.table, rl.column, record)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RowNotFound(table=table, col=column, match=match)
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.254 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.transaction [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
      txn.results.put(txn.do_commit())
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
      command.run_idl(txn)
    File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
      entity = self.api.lookup(self.lookup_table, self.entity)
    File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
      return super().lookup(table, record, default=default, timeout=timeout,
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
      return self._lookup(table, record)
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
      row = idlutils.row_by_value(self, rl.table, rl.column, record)
    File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
      raise RowNotFound(table=table, col=column, match=match)
  ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab

  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Error executing command (PgAclAddCommand): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     t.add(self)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     yield t
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     del self._nested_txns_map[cur_thread_id]
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     command.run_idl(txn)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     entity = self.api.lookup(self.lookup_table, self.entity)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return super().lookup(table, record, default=default, timeout=timeout,
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     return self._lookup(table, record)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     row = idlutils.row_by_value(self, rl.table, rl.column, record)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command     raise RowNotFound(table=table, col=column, match=match)
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.266 1205641 ERROR ovsdbapp.backend.ovs_idl.command 
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Maintenance task: Failed to fix resource e7813967-9b79-4384-b355-bff5c0de88f1 (type: security_group_rules): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 392, in check_for_inconsistencies
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 231, in _fix_create_update
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_create'](context, n_obj)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2269, in create_security_group_rule
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._process_security_group_rule(rule)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2262, in _process_security_group_rule
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_acl.update_acls_for_security_group(
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/common/ovn/acl.py", line 305, in update_acls_for_security_group
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn.pg_acl_add(**acl, may_exist=True).execute(check_error=True)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 42, in execute
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     t.add(self)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 275, in transaction
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     yield t
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 110, in transaction
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     del self._nested_txns_map[cur_thread_id]
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 61, in __exit__
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     command.run_idl(txn)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 117, in run_idl
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     entity = self.api.lookup(self.lookup_table, self.entity)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 207, in lookup
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return super().lookup(table, record, default=default, timeout=timeout,
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     return self._lookup(table, record)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 268, in _lookup
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     row = idlutils.row_by_value(self, rl.table, rl.column, record)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RowNotFound(table=table, col=column, match=match)
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_13f37b64_2c24_48b7_bf76_a04890898cab
  2022-12-27 10:31:07.267 1205641 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
  2022-12-27 10:31:07.268 1205641 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [req-5157728d-d6d9-40ee-8fc4-ef2531f1a16f - - - - -] Maintenance task: Synchronization finished (took 0.07 seconds)
  2022-12-27 10:31:09.358 1205633 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-71a3cffe-7e68-44f7-b31d-aa0d5f8b09f1 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:09.359 1205632 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-add99db6-d820-4a04-9fe6-b08b57884c85 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:09.359 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [req-46908903-6ee7-4a7d-82da-3934a51e1190 - - - - -] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:09.363 1205630 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:09.367 1205635 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'ecc0cc97-5640-40e0-8614-0f17047d4ef3'". This should never happen in a normal situation, please check the status of your cluster
  2022-12-27 10:31:09.375 1205631 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovsdb_monitor [-] HashRing is empty, error: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster: neutron.common.ovn.exceptions.HashRingIsEmpty: Hash Ring returned empty when hashing "b'a805e3ae-0e47-4908-b1c5-9761ec25b2f0'". This should never happen in a normal situation, please check the status of your cluster

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2000495/+subscriptions
Follow ups

[Bug 2000495] Re: See error "HashRingIsEmpty" when create instance
From: Launchpad Bug Tracker, 2023-02-26