← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1939704] [NEW] db-sync script may fail when migrating from ovs to ovn

 

Public bug reported:

During the steps where port bindings are updated [1] there might be a
neutron-server l3 gateway scheduling going on and the port binding for
gateway router ports can change its gateway chassis. It makes the db
sync script fail with following traceback

2021-08-12 06:21:40.738 48 INFO networking_ovn.cmd.neutron_ovn_db_sync_util [req-988ef422-ae85-41d7-a4a0-1e7b6c68a29e - - - - -] Migrating Neutron database from OVS to OVN
2021-08-12 06:21:41.300 48 CRITICAL neutron_ovn_db_sync_util [req-988ef422-ae85-41d7-a4a0-1e7b6c68a29e - - - - -] Unhandled error: neutron_lib.exceptions.ObjectNotFound: Object PortBinding(port_id=a95eaa4b-fe5f-4770-9a03-2e76f20c2870, host=controller-2.redhat.local) not found.
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/bin/neutron-ovn-db-sync-util", line 10, in <module>
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     sys.exit(main())
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/networking_ovn/cmd/neutron_ovn_db_sync_util.py", line 233, in main
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     db_migration.migrate_neutron_database_to_ovn(core_plugin)
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/networking_ovn/ml2/db_migration.py", line 66, in migrate_neutron_database_to_ovn
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     pb.update()
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/base.py", line 337, in decorator
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     return func(self, *args, **kwargs)
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/base.py", line 906, in update
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     self._get_composite_keys()))
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 86, in update_object
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     db_obj = _safe_get_object(obj_cls, context, **kwargs)
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 80, in _safe_get_object
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     id="%s(%s)" % (obj_cls.db_model.__name__, key))
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util neutron_lib.exceptions.ObjectNotFound: Object PortBinding(port_id=a95eaa4b-fe5f-4770-9a03-2e76f20c2870, host=controller-2.redhat.local) not found.
2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util 

[1]
https://opendev.org/openstack/neutron/src/commit/caac890c8e121e1bebe33f4e93ab79d4c294db35/neutron/plugins/ml2/drivers/ovn/db_migration.py#L73-L83

** Affects: neutron
     Importance: Undecided
     Assignee: Jakub Libosvar (libosvar)
         Status: New


** Tags: ovn

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1939704

Title:
  db-sync script may fail when migrating from ovs to ovn

Status in neutron:
  New

Bug description:
  During the steps where port bindings are updated [1] there might be a
  neutron-server l3 gateway scheduling going on and the port binding for
  gateway router ports can change its gateway chassis. It makes the db
  sync script fail with following traceback

  2021-08-12 06:21:40.738 48 INFO networking_ovn.cmd.neutron_ovn_db_sync_util [req-988ef422-ae85-41d7-a4a0-1e7b6c68a29e - - - - -] Migrating Neutron database from OVS to OVN
  2021-08-12 06:21:41.300 48 CRITICAL neutron_ovn_db_sync_util [req-988ef422-ae85-41d7-a4a0-1e7b6c68a29e - - - - -] Unhandled error: neutron_lib.exceptions.ObjectNotFound: Object PortBinding(port_id=a95eaa4b-fe5f-4770-9a03-2e76f20c2870, host=controller-2.redhat.local) not found.
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util Traceback (most recent call last):
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/bin/neutron-ovn-db-sync-util", line 10, in <module>
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     sys.exit(main())
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/networking_ovn/cmd/neutron_ovn_db_sync_util.py", line 233, in main
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     db_migration.migrate_neutron_database_to_ovn(core_plugin)
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/networking_ovn/ml2/db_migration.py", line 66, in migrate_neutron_database_to_ovn
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     pb.update()
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/base.py", line 337, in decorator
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     return func(self, *args, **kwargs)
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/base.py", line 906, in update
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     self._get_composite_keys()))
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 86, in update_object
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     db_obj = _safe_get_object(obj_cls, context, **kwargs)
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util   File "/usr/lib/python3.6/site-packages/neutron/objects/db/api.py", line 80, in _safe_get_object
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util     id="%s(%s)" % (obj_cls.db_model.__name__, key))
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util neutron_lib.exceptions.ObjectNotFound: Object PortBinding(port_id=a95eaa4b-fe5f-4770-9a03-2e76f20c2870, host=controller-2.redhat.local) not found.
  2021-08-12 06:21:41.300 48 ERROR neutron_ovn_db_sync_util 

  [1]
  https://opendev.org/openstack/neutron/src/commit/caac890c8e121e1bebe33f4e93ab79d4c294db35/neutron/plugins/ml2/drivers/ovn/db_migration.py#L73-L83

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1939704/+subscriptions



Follow ups