yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1825088] [NEW] [port forwarding] should not process port forwarding if snat node only run DHCP

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: LIU Yulong <yulong@xxxxxxxxxxxxx>
Date: Wed, 17 Apr 2019 05:09:53 -0000
Reply-to: Bug 1825088 <1825088@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

[port forwarding] should not process port forwarding if snat node only
run DHCP

Assuming you have 3 network nodes, the agent modes are all `dvr_snat`.
One `dvr_ha` router1 is scheduled to node1 and node2. The dhcp namespace
(connected to the router1) is scheduled to node3. Then snat-namespace
will only exist on node1 and node2. But on onde3, the l3 agent will also
process this router because of the DHCP namespace, as well as the
port_forwarding extension. Then exception raised.

Log trace:
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent [req-6507e5bd-a500-47ea-9f71-e0123eb24e63 - a890d8d8264640ba9bae20d03e4071fd - - -] Failed to process compatible router: 867e1473-4495-4513-8759-dee4cb1b9cef: AttributeError: 'NoneType' object has no attribute 'ipv4'
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent Traceback (most recent call last):
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 655, in _process_router_update
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 573, in _process_router_if_compatible
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self._process_added_router(router)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/l3_agent_with_metering.py", line 524, in wrapped
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     func(self, *args, **kwargs)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/l3_agent_with_metering.py", line 532, in _process_added_router
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self)._process_added_router(router)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 583, in _process_added_router
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self.l3_ext_manager.add_router(self.context, router)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/l3_agent_extensions_manager.py", line 42, in add_router
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     extension.obj.add_router(context, data)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 453, in add_router
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self.process_port_forwarding(context, data)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 443, in process_port_forwarding
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     context, ri, ri.fip_managed_by_port_forwardings)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 433, in check_local_port_forwardings
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     namespace, iptable_manager)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "<string>", line 2, in _process_create
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/coordination.py", line 76, in _synchronized
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     return f(*a, **k)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 241, in _process_create
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self._rule_apply(iptables_manager, port_forwarding, rule_tag)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 173, in _rule_apply
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     iptables_manager.ipv4['nat'].clear_rules_by_tag(rule_tag)
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'ipv4'
2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1825088

Title:
  [port forwarding] should not process port forwarding if snat node only
  run DHCP

Status in neutron:
  New

Bug description:
  [port forwarding] should not process port forwarding if snat node only
  run DHCP

  Assuming you have 3 network nodes, the agent modes are all `dvr_snat`.
  One `dvr_ha` router1 is scheduled to node1 and node2. The dhcp
  namespace (connected to the router1) is scheduled to node3. Then snat-
  namespace will only exist on node1 and node2. But on onde3, the l3
  agent will also process this router because of the DHCP namespace, as
  well as the port_forwarding extension. Then exception raised.

  Log trace:
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent [req-6507e5bd-a500-47ea-9f71-e0123eb24e63 - a890d8d8264640ba9bae20d03e4071fd - - -] Failed to process compatible router: 867e1473-4495-4513-8759-dee4cb1b9cef: AttributeError: 'NoneType' object has no attribute 'ipv4'
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent Traceback (most recent call last):
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 655, in _process_router_update
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self._process_router_if_compatible(router)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 573, in _process_router_if_compatible
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self._process_added_router(router)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/l3_agent_with_metering.py", line 524, in wrapped
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     func(self, *args, **kwargs)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/l3_agent_with_metering.py", line 532, in _process_added_router
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self)._process_added_router(router)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py", line 583, in _process_added_router
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self.l3_ext_manager.add_router(self.context, router)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/l3_agent_extensions_manager.py", line 42, in add_router
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     extension.obj.add_router(context, data)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 453, in add_router
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self.process_port_forwarding(context, data)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 443, in process_port_forwarding
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     context, ri, ri.fip_managed_by_port_forwardings)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 433, in check_local_port_forwardings
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     namespace, iptable_manager)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "<string>", line 2, in _process_create
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/coordination.py", line 76, in _synchronized
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     return f(*a, **k)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 241, in _process_create
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     self._rule_apply(iptables_manager, port_forwarding, rule_tag)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent   File "/usr/lib/python2.7/site-packages/neutron/agent/l3/extensions/port_forwarding.py", line 173, in _rule_apply
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent     iptables_manager.ipv4['nat'].clear_rules_by_tag(rule_tag)
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'ipv4'
  2019-04-17 12:39:53.064 3823374 ERROR neutron.agent.l3.agent

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1825088/+subscriptions
Follow ups

[Bug 1825088] Re: [port forwarding] should not process port forwarding if snat node only run DHCP
From: OpenStack Infra, 2019-04-28