yahoo-eng-team team mailing list archive

[Miro Tomaska <2008943@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Runtime exception:
 
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_aa9f203b_ec51_4893_9bda_cfadbff9f800

can occure while performing database sync between Neutron db and OVN NB db using neutron-ovn-db-sync-util.
This exception occures when the `sync_networks_ports_and_dhcp_opts()` function ends up implicitly creating a new default security group for a tenant/project id. This is normally ok but the problem is that `sync_port_groups` was already called and thus the port_group does not exists in NB DB. When the `sync_acls()` is called later there is no port group found and exception occurs.
 
Quick way to reproduce on ML2/OVN:
- openstack create network --project <project_uuid>
- openstack security group delete <default_sg_uuid_belonging_to_that_project>
- neutron-ovn-db-sync-util --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --ovn-neutron_sync_mode migrate


Here is a real world scenario how to run into this problem, including why the code runs into it.
1. ML2/OVS enviroment with a network but no default security group for the project/tenant associated with the network
2. Perform ML2/OVS to ML2/OVN migration. This process will start neutron-ovn-db-sync-util sync
3. During the sync we first sync port groups[1] from Neutron DB to OVN DB
4. Then we sync network ports [2]. The process will detect that the network in question is not part of OVN NB. It will create that network in OVN NB db and along with that it will create a metadata port for it(OVN network requires metadataport). The Port_create call will implicitly notify _ensure_default_security_group_handler which will not find securty group for that tenant/project id and create one. Lastly, new security group will create 4 new default security group rules.
5. When sync_acls[4] runs it will pick up those 4 new rules but commit to NB DB will fail since the port_group(aka security group) does not exists


[1] https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py#L104
[2] https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py#L10
[3] https://opendev.org/openstack/neutron/src/branch/master/neutron/db/securitygroups_db.py#L915
[4] https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py#L107

** Affects: neutron
     Importance: Undecided
     Assignee: Miro Tomaska (mtomaska)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Miro Tomaska (mtomaska)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2008943

Title:
  OVN DB Sync utility cannot find NB DB Port Group

Status in neutron:
  New

Bug description:
  Runtime exception:
   
  ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Port_Group with name=pg_aa9f203b_ec51_4893_9bda_cfadbff9f800

  can occure while performing database sync between Neutron db and OVN NB db using neutron-ovn-db-sync-util.
  This exception occures when the `sync_networks_ports_and_dhcp_opts()` function ends up implicitly creating a new default security group for a tenant/project id. This is normally ok but the problem is that `sync_port_groups` was already called and thus the port_group does not exists in NB DB. When the `sync_acls()` is called later there is no port group found and exception occurs.
   
  Quick way to reproduce on ML2/OVN:
  - openstack create network --project <project_uuid>
  - openstack security group delete <default_sg_uuid_belonging_to_that_project>
  - neutron-ovn-db-sync-util --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --ovn-neutron_sync_mode migrate

  
  Here is a real world scenario how to run into this problem, including why the code runs into it.
  1. ML2/OVS enviroment with a network but no default security group for the project/tenant associated with the network
  2. Perform ML2/OVS to ML2/OVN migration. This process will start neutron-ovn-db-sync-util sync
  3. During the sync we first sync port groups[1] from Neutron DB to OVN DB
  4. Then we sync network ports [2]. The process will detect that the network in question is not part of OVN NB. It will create that network in OVN NB db and along with that it will create a metadata port for it(OVN network requires metadataport). The Port_create call will implicitly notify _ensure_default_security_group_handler which will not find securty group for that tenant/project id and create one. Lastly, new security group will create 4 new default security group rules.
  5. When sync_acls[4] runs it will pick up those 4 new rules but commit to NB DB will fail since the port_group(aka security group) does not exists

  
  [1] https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py#L104
  [2] https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py#L10
  [3] https://opendev.org/openstack/neutron/src/branch/master/neutron/db/securitygroups_db.py#L915
  [4] https://opendev.org/openstack/neutron/src/branch/master/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py#L107

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2008943/+subscriptions