yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1946624] [NEW] OVSDB Error: Transaction causes multiple rows in "Port_Group" table to have identical values

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sam Wan <1946624@xxxxxxxxxxxxxxxxxx>
Date: Mon, 11 Oct 2021 08:12:23 -0000
Reply-to: Bug 1946624 <1946624@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

devstack failed due to devstack@q-svc failed with below errors
===============================
Oct 09 08:37:22.122658 e2e-os-xiofc116 neutron-server[1219129]: DEBUG neutron.common.config [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=1219129) log_opt_values
/usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Oct 09 08:37:35.689569 e2e-os-xiofc116 neutron-server[1219129]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: {"details":"Transaction causes multiple rows in \"Port_Group\" t
able to have identical values (neutron_pg_drop) for index on column \"name\".  First row, with UUID f8889a2b-2a32-4f62-b000-eed67c6a11f4, existed in the database before this transaction an
d was not modified by the transaction.  Second row, with UUID 32a90604-ba30-43fe-b12f-12d372231faf, was inserted by this transaction.","error":"constraint violation"}
Oct 09 08:37:35.690362 e2e-os-xiofc116 neutron-server[1219129]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e00ddfe7-ecf7-4a23-b158-b94229bb528d None None] Traceback (most recent
call last):
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e00ddfe7-ecf7-4a23-b158-b94229bb528d None Non
e] OVS database connection to OVN_Southbound failed with error: ''>' not supported between instances of 'Row' and 'Atom''. Verify that the OVS and OVN services are available and that the '
ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: TypeError: '>' not supported between instances of 'Row' and 'Atom'
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/opt/stack/new/neutron/neutron/plugins/ml2/driv
ers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 127, in start_connection
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.ovsdb_connection.start()
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovsdbapp
/backend/ovs_idl/connection.py", line 83, in start
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     idlutils.wait_for_change(self.idl, self.timeout)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/opt/stack/new/neutron/neutron/plugins/ml2/driv
ers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 53, in wait_for_change
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     while idl_.change_seqno == seqno and not idl_.run()
:
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/i
dl.py", line 269, in run
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.__parse_update(msg.params[1], OVSDB_UPDATE2)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/i
dl.py", line 610, in __parse_update
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.__do_parse_update(update, version, self.tables
)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/i
dl.py", line 648, in __do_parse_update
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     changes = self.__process_update2(table, uuid, row_u
pdate)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 693, in __process_update2
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     table.rows[uuid] = row
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/custom_index.py", line 96, in __setitem__
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     index.add(item)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/custom_index.py", line 63, in add
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.values.add(self.index_entry_from_row(row))
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/sortedcontainers/sortedlist.py", line 1801, in add
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     pos = bisect_right(_maxes, key)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/custom_index.py", line 49, in _cmp
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     result = (aval > bval) - (aval < bval)
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn TypeError: '>' not supported between instances of 'Row' and 'Atom'
Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron Traceback (most recent call last):
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/bin/neutron-server", line 10, in <module>
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     sys.exit(main())
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/cmd/eventlet/server/__init__.py", line 19, in main
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     server.boot_server(wsgi_eventlet.eventlet_wsgi_server)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/server/__init__.py", line 68, in boot_server
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     server_func()
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/server/wsgi_eventlet.py", line 24, in eventlet_wsgi_server
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     neutron_api = service.serve_wsgi(service.NeutronApiService)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/service.py", line 94, in serve_wsgi
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     registry.publish(resources.PROCESS, events.BEFORE_SPAWN, service)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/registry.py", line 60, in publish
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     _get_callback_manager().publish(resource, event, trigger, payload=payload)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py", line 149, in publish
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     return self.notify(resource, event, trigger, payload=payload)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/db/utils.py", line 110, in _wrapped
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     raise db_exc.RetryRequest(e)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     self.force_reraise()
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     raise self.value
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/db/utils.py", line 105, in _wrapped
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     return function(*args, **kwargs)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py", line 174, in notify
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     raise exceptions.CallbackFailure(errors=errors)
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_dr
iver.OVNMechanismDriver.pre_fork_initialize-1513560 failed with "OVS database connection to OVN_Southbound failed with error: ''>' not supported between instances of 'Row' and 'Atom''. Ver
ify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct."
Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron
==========================

for more detailed logs, plesae check: https://elab-os-
logsrv.delllabs.net/19/813219/1/check/DellEMC_XtremIO_FC/5df2b13/DellEMC_XtremIO_FC/20/logs

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1946624

Title:
  OVSDB Error: Transaction causes multiple rows in "Port_Group" table to
  have identical values

Status in neutron:
  New

Bug description:
  devstack failed due to devstack@q-svc failed with below errors
  ===============================
  Oct 09 08:37:22.122658 e2e-os-xiofc116 neutron-server[1219129]: DEBUG neutron.common.config [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=1219129) log_opt_values
  /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
  Oct 09 08:37:35.689569 e2e-os-xiofc116 neutron-server[1219129]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: {"details":"Transaction causes multiple rows in \"Port_Group\" t
  able to have identical values (neutron_pg_drop) for index on column \"name\".  First row, with UUID f8889a2b-2a32-4f62-b000-eed67c6a11f4, existed in the database before this transaction an
  d was not modified by the transaction.  Second row, with UUID 32a90604-ba30-43fe-b12f-12d372231faf, was inserted by this transaction.","error":"constraint violation"}
  Oct 09 08:37:35.690362 e2e-os-xiofc116 neutron-server[1219129]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-e00ddfe7-ecf7-4a23-b158-b94229bb528d None None] Traceback (most recent
  call last):
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-e00ddfe7-ecf7-4a23-b158-b94229bb528d None Non
  e] OVS database connection to OVN_Southbound failed with error: ''>' not supported between instances of 'Row' and 'Atom''. Verify that the OVS and OVN services are available and that the '
  ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct.: TypeError: '>' not supported between instances of 'Row' and 'Atom'
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most recent call last):
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/opt/stack/new/neutron/neutron/plugins/ml2/driv
  ers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 127, in start_connection
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.ovsdb_connection.start()
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovsdbapp
  /backend/ovs_idl/connection.py", line 83, in start
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     idlutils.wait_for_change(self.idl, self.timeout)
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/opt/stack/new/neutron/neutron/plugins/ml2/driv
  ers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 53, in wait_for_change
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     while idl_.change_seqno == seqno and not idl_.run()
  :
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/i
  dl.py", line 269, in run
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.__parse_update(msg.params[1], OVSDB_UPDATE2)
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/i
  dl.py", line 610, in __parse_update
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.__do_parse_update(update, version, self.tables
  )
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/i
  dl.py", line 648, in __do_parse_update
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     changes = self.__process_update2(table, uuid, row_u
  pdate)
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 693, in __process_update2
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     table.rows[uuid] = row
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/custom_index.py", line 96, in __setitem__
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     index.add(item)
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/custom_index.py", line 63, in add
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     self.values.add(self.index_entry_from_row(row))
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/sortedcontainers/sortedlist.py", line 1801, in add
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     pos = bisect_right(_maxes, key)
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn   File "/usr/local/lib/python3.8/dist-packages/ovs/db/custom_index.py", line 49, in _cmp
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn     result = (aval > bval) - (aval < bval)
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn TypeError: '>' not supported between instances of 'Row' and 'Atom'
  Oct 09 08:39:06.661299 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron Traceback (most recent call last):
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/bin/neutron-server", line 10, in <module>
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     sys.exit(main())
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/cmd/eventlet/server/__init__.py", line 19, in main
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     server.boot_server(wsgi_eventlet.eventlet_wsgi_server)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/server/__init__.py", line 68, in boot_server
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     server_func()
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/server/wsgi_eventlet.py", line 24, in eventlet_wsgi_server
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     neutron_api = service.serve_wsgi(service.NeutronApiService)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/opt/stack/new/neutron/neutron/service.py", line 94, in serve_wsgi
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     registry.publish(resources.PROCESS, events.BEFORE_SPAWN, service)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/registry.py", line 60, in publish
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     _get_callback_manager().publish(resource, event, trigger, payload=payload)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py", line 149, in publish
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     return self.notify(resource, event, trigger, payload=payload)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/db/utils.py", line 110, in _wrapped
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     raise db_exc.RetryRequest(e)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     self.force_reraise()
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     raise self.value
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/db/utils.py", line 105, in _wrapped
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     return function(*args, **kwargs)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron   File "/usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py", line 174, in notify
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron     raise exceptions.CallbackFailure(errors=errors)
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_dr
  iver.OVNMechanismDriver.pre_fork_initialize-1513560 failed with "OVS database connection to OVN_Southbound failed with error: ''>' not supported between instances of 'Row' and 'Atom''. Ver
  ify that the OVS and OVN services are available and that the 'ovn_nb_connection' and 'ovn_sb_connection' configuration options are correct."
  Oct 09 08:39:06.663456 e2e-os-xiofc116 neutron-server[1219129]: ERROR neutron
  ==========================

  for more detailed logs, plesae check: https://elab-os-
  logsrv.delllabs.net/19/813219/1/check/DellEMC_XtremIO_FC/5df2b13/DellEMC_XtremIO_FC/20/logs

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1946624/+subscriptions
Follow ups

[Bug 1946624] Re: OVSDB Error: Transaction causes multiple rows in "Port_Group" table to have identical values
From: Sam Wan, 2021-11-24