yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1905552] [NEW] neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown'

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Zhang Jian <1905552@xxxxxxxxxxxxxxxxxx>
Date: Wed, 25 Nov 2020 11:17:06 -0000
Reply-to: Bug 1905552 <1905552@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

2020-11-25 11:07:32.606 127 DEBUG oslo_concurrency.lockutils [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Releasing semaphore "iptables-qrouter-9e18395d-961d-46b3-a0e9-4c6a94c32baf" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:228
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Failed to update firewall: daedc38a-04ee-4818-b7a6-3d8311d7fc30: KeyError: 'unknown'
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 Traceback (most recent call last):
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 144, in update_firewall_group
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     apply_list, self.pre_firewall, firewall)
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 327, in _remove_conntrack_updated_firewall
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     ipt_mgr.namespace)
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/netlink_conntrack.py", line 41, in delete_entries
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     entries = nl_lib.list_entries(namespace)
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 207, in _wrap
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     return self.channel.remote_call(name, args, kwargs)
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 202, in remote_call
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     raise exc_type(*result[2])
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 KeyError: 'unknown'
2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2

This error appears when  configured the neutron-fwaas v2 with netlink_conntrack driver in fwaas_agent.ini
vim /etc/kolla/neutron-l3-agent/fwaas_driver.ini 
   [fwaas]
   enabled = True
   agent_version = v2
   driver = iptables_v2
   conntrack_driver = netlink_conntrack

And the conntrack list has 'unknown' rules, example below:
unknown  2 597 src=169.254.192.2 dst=224.0.0.22 [UNREPLIED] src=224.0.0.22 dst=169.254.192.2 mark=0 use=1
unknown  112 598 src=169.254.192.2 dst=224.0.0.18 [UNREPLIED] src=224.0.0.18 dst=169.254.192.2 mark=0 use=1

This may interrupt conntrack refresh when firewall rules update.

** Affects: neutron
     Importance: Undecided
     Assignee: Zhang Jian (jasonzhangj)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Zhang Jian (q5536487)

** Changed in: neutron
     Assignee: Zhang Jian (jasonzhangj) => (unassigned)

** Changed in: neutron
     Assignee: (unassigned) => Zhang Jian (jasonzhangj)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1905552

Title:
  neutron-fwaas netlink conntrack driver would catch error while
  conntrack rules protocol is 'unknown'

Status in neutron:
  New

Bug description:
  2020-11-25 11:07:32.606 127 DEBUG oslo_concurrency.lockutils [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Releasing semaphore "iptables-qrouter-9e18395d-961d-46b3-a0e9-4c6a94c32baf" lock /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:228
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 [req-ab14782d-80b1-43f6-8d1b-2874531aca5e - 9d40b483f885496896d81c487f420438 - - -] Failed to update firewall: daedc38a-04ee-4818-b7a6-3d8311d7fc30: KeyError: 'unknown'
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 Traceback (most recent call last):
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 144, in update_firewall_group
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     apply_list, self.pre_firewall, firewall)
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/iptables_fwaas_v2.py", line 327, in _remove_conntrack_updated_firewall
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     ipt_mgr.namespace)
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/neutron_fwaas/services/firewall/service_drivers/agents/drivers/linux/netlink_conntrack.py", line 41, in delete_entries
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     entries = nl_lib.list_entries(namespace)
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 207, in _wrap
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     return self.channel.remote_call(name, args, kwargs)
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2   File "/var/lib/kolla/venv/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 202, in remote_call
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2     raise exc_type(*result[2])
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2 KeyError: 'unknown'
  2020-11-25 11:07:32.609 127 ERROR neutron_fwaas.services.firewall.service_drivers.agents.drivers.linux.iptables_fwaas_v2

  This error appears when  configured the neutron-fwaas v2 with netlink_conntrack driver in fwaas_agent.ini
  vim /etc/kolla/neutron-l3-agent/fwaas_driver.ini 
     [fwaas]
     enabled = True
     agent_version = v2
     driver = iptables_v2
     conntrack_driver = netlink_conntrack

  And the conntrack list has 'unknown' rules, example below:
  unknown  2 597 src=169.254.192.2 dst=224.0.0.22 [UNREPLIED] src=224.0.0.22 dst=169.254.192.2 mark=0 use=1
  unknown  112 598 src=169.254.192.2 dst=224.0.0.18 [UNREPLIED] src=224.0.0.18 dst=169.254.192.2 mark=0 use=1

  This may interrupt conntrack refresh when firewall rules update.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1905552/+subscriptions
Follow ups

[Bug 1905552] Re: neutron-fwaas netlink conntrack driver would catch error while conntrack rules protocol is 'unknown'
From: Oleg Bondarev, 2020-11-25