yahoo-eng-team team mailing list archive

[venkata anil <anil.venkata@xxxxxxxxxxxx>]

Public bug reported:

While allocating or updating ips for port, _get_subnet and _get_subents
are called multiple times.

For example, if update_port is called with below fixed_ips
fixed_ips = [{'subnet_id': 'subnet1'},
             {'subnet_id': 'v6_dhcp_stateless_subnet'}
             {'subnet_id': 'v6_slaac_subnet'}
             {'subnet_id': 'v6_pd_subnet'}
             {'subnet_id': 'subnet4', 'ip_address': '30.0.0.3'}
             {'ip_address': '40.0.0.4'}
             {'ip_address': '50.0.0.5'}}
then through _test_fixed_ips_for_port(fixed_ips),  "_get_subnet"[1] is called 4 times for subnet1, v6_dhcp_stateless_subnet, v6_slaac_subnet, v6_pd_subnet, subnet4. "_get_subnets" [2] is called 2 times for ip_address 40.0.0.4 and 50.0.0.5.


If in case of _test_fixed_ips_for_port called for _allocate_ips_for_port, then _get_subnets is already called at[3] (so increase call count). So incase of _allocate_ips_for_port, if we save subnets from [3] saved in local variable and use that in-memory subnets in further calls, we can avoid above calls to DB.


Sometimes when subnet is updated, update_subnet may trigger update_port(fixed_ips)[4] for all ports on the subnet. And in each port's fixed_ips, if we have multiple subnets and ip_addresses then _get_subnet and _get_subnets will be called for multiple times for each port like above example. For example in above case if we have 10 ports on the subnet, then update_subnet will result in (10*6=60) 60 times DB access instead of 10 DB access. 


When port_update is called for PD subnet, it again calls get_subnet for each fixed_ip[5], to check if subnet is PD subnet or not(after get_subnet and get_subnets called many times in _test_fixed_ips_for_port).


In all above cases, for _get_subnet and _get_subnets, we are accessing DB many times.
 So instead of calling get_subnet or get_subnets for each fixed_ip of a port(at multiple places), we can call get_subnets of a network at begining of _allocate_ips_for_port(for create port) and _update_ips_for_port(during update) and use the in-memory subnets in subsequent private functions.


[1] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L311
[2] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L331
[3] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_pluggable_backend.py#L192
[4] https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L785
[5] https://review.openstack.org/#/c/241227/11/neutron/db/ipam_non_pluggable_backend.py Lines 284 and 334.

** Affects: neutron
     Importance: Undecided
     Assignee: venkata anil (anil-venkata)
         Status: New


** Tags: ipv6 l3-ipam-dhcp

** Changed in: neutron
     Assignee: (unassigned) => venkata anil (anil-venkata)

** Tags added: ipv6 l3-ipam-dhcp

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1554414

Title:
  Avoid calling _get_subnet(s) multiple times in ipam driver

Status in neutron:
  New

Bug description:
  While allocating or updating ips for port, _get_subnet and
  _get_subents are called multiple times.

  For example, if update_port is called with below fixed_ips
  fixed_ips = [{'subnet_id': 'subnet1'},
               {'subnet_id': 'v6_dhcp_stateless_subnet'}
               {'subnet_id': 'v6_slaac_subnet'}
               {'subnet_id': 'v6_pd_subnet'}
               {'subnet_id': 'subnet4', 'ip_address': '30.0.0.3'}
               {'ip_address': '40.0.0.4'}
               {'ip_address': '50.0.0.5'}}
  then through _test_fixed_ips_for_port(fixed_ips),  "_get_subnet"[1] is called 4 times for subnet1, v6_dhcp_stateless_subnet, v6_slaac_subnet, v6_pd_subnet, subnet4. "_get_subnets" [2] is called 2 times for ip_address 40.0.0.4 and 50.0.0.5.

  
  If in case of _test_fixed_ips_for_port called for _allocate_ips_for_port, then _get_subnets is already called at[3] (so increase call count). So incase of _allocate_ips_for_port, if we save subnets from [3] saved in local variable and use that in-memory subnets in further calls, we can avoid above calls to DB.

  
  Sometimes when subnet is updated, update_subnet may trigger update_port(fixed_ips)[4] for all ports on the subnet. And in each port's fixed_ips, if we have multiple subnets and ip_addresses then _get_subnet and _get_subnets will be called for multiple times for each port like above example. For example in above case if we have 10 ports on the subnet, then update_subnet will result in (10*6=60) 60 times DB access instead of 10 DB access. 

  
  When port_update is called for PD subnet, it again calls get_subnet for each fixed_ip[5], to check if subnet is PD subnet or not(after get_subnet and get_subnets called many times in _test_fixed_ips_for_port).

  
  In all above cases, for _get_subnet and _get_subnets, we are accessing DB many times.
   So instead of calling get_subnet or get_subnets for each fixed_ip of a port(at multiple places), we can call get_subnets of a network at begining of _allocate_ips_for_port(for create port) and _update_ips_for_port(during update) and use the in-memory subnets in subsequent private functions.

  
  [1] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L311
  [2] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L331
  [3] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_pluggable_backend.py#L192
  [4] https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L785
  [5] https://review.openstack.org/#/c/241227/11/neutron/db/ipam_non_pluggable_backend.py Lines 284 and 334.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1554414/+subscriptions