yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #47613
[Bug 1554414] [NEW] Avoid calling _get_subnet(s) multiple times in ipam driver
Public bug reported:
While allocating or updating ips for port, _get_subnet and _get_subents
are called multiple times.
For example, if update_port is called with below fixed_ips
fixed_ips = [{'subnet_id': 'subnet1'},
{'subnet_id': 'v6_dhcp_stateless_subnet'}
{'subnet_id': 'v6_slaac_subnet'}
{'subnet_id': 'v6_pd_subnet'}
{'subnet_id': 'subnet4', 'ip_address': '30.0.0.3'}
{'ip_address': '40.0.0.4'}
{'ip_address': '50.0.0.5'}}
then through _test_fixed_ips_for_port(fixed_ips), "_get_subnet"[1] is called 4 times for subnet1, v6_dhcp_stateless_subnet, v6_slaac_subnet, v6_pd_subnet, subnet4. "_get_subnets" [2] is called 2 times for ip_address 40.0.0.4 and 50.0.0.5.
If in case of _test_fixed_ips_for_port called for _allocate_ips_for_port, then _get_subnets is already called at[3] (so increase call count). So incase of _allocate_ips_for_port, if we save subnets from [3] saved in local variable and use that in-memory subnets in further calls, we can avoid above calls to DB.
Sometimes when subnet is updated, update_subnet may trigger update_port(fixed_ips)[4] for all ports on the subnet. And in each port's fixed_ips, if we have multiple subnets and ip_addresses then _get_subnet and _get_subnets will be called for multiple times for each port like above example. For example in above case if we have 10 ports on the subnet, then update_subnet will result in (10*6=60) 60 times DB access instead of 10 DB access.
When port_update is called for PD subnet, it again calls get_subnet for each fixed_ip[5], to check if subnet is PD subnet or not(after get_subnet and get_subnets called many times in _test_fixed_ips_for_port).
In all above cases, for _get_subnet and _get_subnets, we are accessing DB many times.
So instead of calling get_subnet or get_subnets for each fixed_ip of a port(at multiple places), we can call get_subnets of a network at begining of _allocate_ips_for_port(for create port) and _update_ips_for_port(during update) and use the in-memory subnets in subsequent private functions.
[1] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L311
[2] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L331
[3] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_pluggable_backend.py#L192
[4] https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L785
[5] https://review.openstack.org/#/c/241227/11/neutron/db/ipam_non_pluggable_backend.py Lines 284 and 334.
** Affects: neutron
Importance: Undecided
Assignee: venkata anil (anil-venkata)
Status: New
** Tags: ipv6 l3-ipam-dhcp
** Changed in: neutron
Assignee: (unassigned) => venkata anil (anil-venkata)
** Tags added: ipv6 l3-ipam-dhcp
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1554414
Title:
Avoid calling _get_subnet(s) multiple times in ipam driver
Status in neutron:
New
Bug description:
While allocating or updating ips for port, _get_subnet and
_get_subents are called multiple times.
For example, if update_port is called with below fixed_ips
fixed_ips = [{'subnet_id': 'subnet1'},
{'subnet_id': 'v6_dhcp_stateless_subnet'}
{'subnet_id': 'v6_slaac_subnet'}
{'subnet_id': 'v6_pd_subnet'}
{'subnet_id': 'subnet4', 'ip_address': '30.0.0.3'}
{'ip_address': '40.0.0.4'}
{'ip_address': '50.0.0.5'}}
then through _test_fixed_ips_for_port(fixed_ips), "_get_subnet"[1] is called 4 times for subnet1, v6_dhcp_stateless_subnet, v6_slaac_subnet, v6_pd_subnet, subnet4. "_get_subnets" [2] is called 2 times for ip_address 40.0.0.4 and 50.0.0.5.
If in case of _test_fixed_ips_for_port called for _allocate_ips_for_port, then _get_subnets is already called at[3] (so increase call count). So incase of _allocate_ips_for_port, if we save subnets from [3] saved in local variable and use that in-memory subnets in further calls, we can avoid above calls to DB.
Sometimes when subnet is updated, update_subnet may trigger update_port(fixed_ips)[4] for all ports on the subnet. And in each port's fixed_ips, if we have multiple subnets and ip_addresses then _get_subnet and _get_subnets will be called for multiple times for each port like above example. For example in above case if we have 10 ports on the subnet, then update_subnet will result in (10*6=60) 60 times DB access instead of 10 DB access.
When port_update is called for PD subnet, it again calls get_subnet for each fixed_ip[5], to check if subnet is PD subnet or not(after get_subnet and get_subnets called many times in _test_fixed_ips_for_port).
In all above cases, for _get_subnet and _get_subnets, we are accessing DB many times.
So instead of calling get_subnet or get_subnets for each fixed_ip of a port(at multiple places), we can call get_subnets of a network at begining of _allocate_ips_for_port(for create port) and _update_ips_for_port(during update) and use the in-memory subnets in subsequent private functions.
[1] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L311
[2] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_backend_mixin.py#L331
[3] https://github.com/openstack/neutron/blob/master/neutron/db/ipam_pluggable_backend.py#L192
[4] https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L785
[5] https://review.openstack.org/#/c/241227/11/neutron/db/ipam_non_pluggable_backend.py Lines 284 and 334.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1554414/+subscriptions
Follow ups