yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #86057
[Bug 1850779] Re: [L3] snat-ns will be initialized twice for DVR+HA routers during agent restart
This bug was fixed in the package neutron - 2:12.1.1-0ubuntu7
---------------
neutron (2:12.1.1-0ubuntu7) bionic; urgency=medium
* Handle OVSFWPortNotFound and OVSFWTagNotFound in ovs firewall
- d/p/0001-Handle-OVSFWPortNotFound-and-OVSFWTagNotFound-in-ovs.patch
(LP: #1849098).
neutron (2:12.1.1-0ubuntu6) bionic; urgency=medium
* Do not initialize snat-ns twice (LP: #1850779)
- d/p/0001-Do-not-initialize-snat-ns-twice.patch
neutron (2:12.1.1-0ubuntu5) bionic; urgency=medium
* Backport fix for dvr-snat missig rfp interfaces (LP: #1894843)
- d/p/0001-Fix-deletion-of-rfp-interfaces-when-router-is-re-ena.patch
-- Seyeong Kim <seyeong.kim@xxxxxxxxxxxxx> Mon, 03 May 2021 17:15:28
+0900
** Changed in: neutron (Ubuntu Bionic)
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1850779
Title:
[L3] snat-ns will be initialized twice for DVR+HA routers during agent
restart
Status in neutron:
Fix Released
Status in neutron package in Ubuntu:
Fix Released
Status in neutron source package in Bionic:
Fix Released
Status in neutron source package in Focal:
Fix Released
Status in neutron source package in Groovy:
Fix Released
Status in neutron source package in Hirsute:
Fix Released
Bug description:
If the DVR+HA router has external gateway, the snat-namespace will be initialized twice during agent restart.
And that initialized function will run many [1][2] external resource processing actions which will definitely increase the starting time of agent.
https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_snat_ns.py#L31-L39
https://github.com/openstack/neutron/blob/master/neutron/agent/l3/namespaces.py#L91-L108
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
SRU:
[Impact]
Longer l3-agent initialization time during restarts due to creation of snat namespace and setting corresponding sysctl twice.
With this fix, the initialization phase is triggered only once.
[Test Case]
* deploy Openstack on bionic queens (with neutron dvr l3 ha settings and debug mode on for neutron ) and create a router
(If stsstack-bundles are used, here are the commands
./generate-bundle.sh -s bionic -n bionicqueens --dvr-snat-l3ha --create-model --run
./configure
# Configure creates a router with external gateway attached
)
* Restart neutron-l3-agent on one of the node
systemctl restart neutron-l3-agent.service
* Check /var/log/neutron/neutron-l3-agent.log and wait for the logs to be settled with all initialization steps
During initialization steps, following sysctl's are configured [1] [2].
Verify if the debug logs show sysctl execution statements are displayed twice after restart for snat namespace.
(If the fix is applied they should be displayed only once)
grep -inr snat-<router-id> /var/log/neutron/neutron-l3-agent.log |
grep sysctl
Example log:
2718:2021-04-14 05:17:20.114 10868 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'snat-f64dded1-ef73-47b4-bcee-bb25840e9a02', 'sysctl', '-w', 'net.ipv4.ip_forward=1'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:87
[Where problems could occur]
no regression is expected, but if one occurs it would likely result in longer init time and/or failure to correctly init the snat-namespace
[1] https://github.com/openstack/neutron/blob/master/neutron/agent/l3/dvr_snat_ns.py#L31-L39
[2] https://github.com/openstack/neutron/blob/master/neutron/agent/l3/namespaces.py#L91-L108
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1850779/+subscriptions
References