← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2025202] [NEW] Execute neutron-ovn-db-sync-util report TypeError

 

Public bug reported:

A TypeError was thrown during a synchronization command(eutron-ovn-db-sync-util) execution. By checking the error call stack, it was found that it was an error during the creation of QoS. After analysis, there should be a port in the Neutron database, but not in the ovn-nb database. At this time, when executing the UpdateLSwitchQosOptionsCommand to create a logical port and update the QoS, it was found that the port_id is None. tracking variable  port_id is obtained by executing AddLSwitchPortCommand. It should be that this command did not set the port_id correctly caused. Analyzing AddLSwitchPortCommand, it was found that if the port already exists, no result was set. 
This seems a bit contradictory. It was determined earlier that the port does not exist, but later it does. I think this situation may occur when executing synchronization commands and calling the API that creates the port. This operation is not very reasonable.

But I think the AddLSwitchPortCommand command should return consistent
results. This issue should be fixed.


ERROR Message:

2023-06-26 11:06:24.385 345 WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-01d7864c-e3a6-409a-a852-2f6ea869fdae - - - - -] Port found in Neutron but not in OVN DB, port_id=ae5a8d95-e59f-465a-833d-28b3d0fabb2d
2023-06-26 11:06:24.440 345 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-01d7864c-e3a6-409a-a852-2f6ea869fdae - - - - -] Traceback (most recent call last):
  File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
    txn.results.put(txn.do_commit())
  File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
    command.run_idl(txn)
  File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 216, in run_idl
    port = self.api.lookup('Logical_Switch_Port', port_id)
  File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 181, in lookup
    return self._lookup(table, record)
  File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 203, in _lookup
    uuid_ = uuid.UUID(record)
  File "/usr/lib64/python3.6/uuid.py", line 134, in __init__
    raise TypeError('one of the hex, bytes, bytes_le, fields, '
TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given

2023-06-26 11:06:24.441 345 CRITICAL neutron_ovn_db_sync_util [None req-01d7864c-e3a6-409a-a852-2f6ea869fdae - - - - -] Unhandled error: TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/bin/neutron-ovn-db-sync-util", line 8, in <module>
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     sys.exit(main())
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/cmd/ovn/neutron_ovn_db_sync_util.py", line 231, in main
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     synchronizer.do_sync()
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 104, in do_sync
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self.sync_networks_ports_and_dhcp_opts(ctx)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 999, in sync_networks_ports_and_dhcp_opts
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     for port_id, port in db_ports.items():
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 120, in _create_port_in_ovn
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self._ovn_client.create_port(ctx, port)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 427, in create_port
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self._qos_driver.create_port(txn, port, port_cmd)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     next(self.gen)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 262, in transaction
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     yield t
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     next(self.gen)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/api.py", line 110, in transaction
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     del self._nested_txns_map[cur_thread_id]
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/api.py", line 61, in __exit__
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self.result = self.commit()
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     raise result.ex
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     txn.results.put(txn.do_commit())
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     command.run_idl(txn)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 216, in run_idl
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     port = self.api.lookup('Logical_Switch_Port', port_id)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 181, in lookup
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     return self._lookup(table, record)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 203, in _lookup
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     uuid_ = uuid.UUID(record)
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/usr/lib64/python3.6/uuid.py", line 134, in __init__
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     raise TypeError('one of the hex, bytes, bytes_le, fields, '
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given
2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util

** Affects: neutron
     Importance: Undecided
     Assignee: ZhouHeng (zhouhenglc)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => ZhouHeng (zhouhenglc)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2025202

Title:
  Execute neutron-ovn-db-sync-util  report TypeError

Status in neutron:
  New

Bug description:
  A TypeError was thrown during a synchronization command(eutron-ovn-db-sync-util) execution. By checking the error call stack, it was found that it was an error during the creation of QoS. After analysis, there should be a port in the Neutron database, but not in the ovn-nb database. At this time, when executing the UpdateLSwitchQosOptionsCommand to create a logical port and update the QoS, it was found that the port_id is None. tracking variable  port_id is obtained by executing AddLSwitchPortCommand. It should be that this command did not set the port_id correctly caused. Analyzing AddLSwitchPortCommand, it was found that if the port already exists, no result was set. 
  This seems a bit contradictory. It was determined earlier that the port does not exist, but later it does. I think this situation may occur when executing synchronization commands and calling the API that creates the port. This operation is not very reasonable.

  But I think the AddLSwitchPortCommand command should return consistent
  results. This issue should be fixed.

  
  ERROR Message:

  2023-06-26 11:06:24.385 345 WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-01d7864c-e3a6-409a-a852-2f6ea869fdae - - - - -] Port found in Neutron but not in OVN DB, port_id=ae5a8d95-e59f-465a-833d-28b3d0fabb2d
  2023-06-26 11:06:24.440 345 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-01d7864c-e3a6-409a-a852-2f6ea869fdae - - - - -] Traceback (most recent call last):
    File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
      txn.results.put(txn.do_commit())
    File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
      command.run_idl(txn)
    File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 216, in run_idl
      port = self.api.lookup('Logical_Switch_Port', port_id)
    File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 181, in lookup
      return self._lookup(table, record)
    File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 203, in _lookup
      uuid_ = uuid.UUID(record)
    File "/usr/lib64/python3.6/uuid.py", line 134, in __init__
      raise TypeError('one of the hex, bytes, bytes_le, fields, '
  TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given

  2023-06-26 11:06:24.441 345 CRITICAL neutron_ovn_db_sync_util [None req-01d7864c-e3a6-409a-a852-2f6ea869fdae - - - - -] Unhandled error: TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/bin/neutron-ovn-db-sync-util", line 8, in <module>
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     sys.exit(main())
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/cmd/ovn/neutron_ovn_db_sync_util.py", line 231, in main
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     synchronizer.do_sync()
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 104, in do_sync
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self.sync_networks_ports_and_dhcp_opts(ctx)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 999, in sync_networks_ports_and_dhcp_opts
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     for port_id, port in db_ports.items():
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py", line 120, in _create_port_in_ovn
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self._ovn_client.create_port(ctx, port)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 427, in create_port
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self._qos_driver.create_port(txn, port, port_cmd)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     next(self.gen)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 262, in transaction
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     yield t
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     next(self.gen)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/api.py", line 110, in transaction
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     del self._nested_txns_map[cur_thread_id]
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/api.py", line 61, in __exit__
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     self.result = self.commit()
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     raise result.ex
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     txn.results.put(txn.do_commit())
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 92, in do_commit
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     command.run_idl(txn)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/root/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py", line 216, in run_idl
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     port = self.api.lookup('Logical_Switch_Port', port_id)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 181, in lookup
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     return self._lookup(table, record)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 203, in _lookup
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     uuid_ = uuid.UUID(record)
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util   File "/usr/lib64/python3.6/uuid.py", line 134, in __init__
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util     raise TypeError('one of the hex, bytes, bytes_le, fields, '
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util TypeError: one of the hex, bytes, bytes_le, fields, or int arguments must be given
  2023-06-26 11:06:24.441 345 ERROR neutron_ovn_db_sync_util

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2025202/+subscriptions



Follow ups