yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1719711] [NEW] iptables failed to apply when binding a port with AGENT.debug_iptables_rules enabled

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Ihar Hrachyshka <1719711@xxxxxxxxxxxxxxxxxx>
Date: Tue, 26 Sep 2017 18:48:59 -0000
Reply-to: Bug 1719711 <1719711@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

http://logs.openstack.org/21/504021/2/check/gate-tempest-dsvm-neutron-
scenario-linuxbridge-ubuntu-xenial-nv/e47a3f3/testr_results.html.gz


Traceback (most recent call last):
  File "/opt/stack/new/neutron/neutron/tests/tempest/scenario/test_security_groups.py", line 127, in test_two_sec_groups
    num_servers=1, security_groups=security_groups_list)
  File "/opt/stack/new/neutron/neutron/tests/tempest/scenario/test_security_groups.py", line 54, in create_vm_testing_sec_grp
    const.SERVER_STATUS_ACTIVE)
  File "tempest/common/waiters.py", line 76, in wait_for_server_status
    server_id=server_id)
tempest.exceptions.BuildErrorException: Server e1120d99-f0eb-43eb-a38b-847843a838b5 failed to build and is in ERROR status
Details: {u'message': u'Build of instance e1120d99-f0eb-43eb-a38b-847843a838b5 aborted: Failed to allocate the network(s), not rescheduling.', u'code': 500, u'created': u'2017-09-26T09:23:42Z'}

In linuxbridge agent log: http://logs.openstack.org/21/504021/2/check
/gate-tempest-dsvm-neutron-scenario-linuxbridge-ubuntu-xenial-
nv/e47a3f3/logs/screen-q-agt.txt.gz?level=TRACE#_Sep_26_09_16_30_623747

Sep 26 09:16:30.623747 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.agent.linux.iptables_manager [None req-78fc6bc1-a089-4d5f-91d8-e5191e45978c None None] IPTables Rules did not converge. Diff: # Generated by iptables_manager
Sep 26 09:16:30.623936 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: *filter
Sep 26 09:16:30.624117 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -D neutron-linuxbri-ibc1a22b9-e 6
Sep 26 09:16:30.624316 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN
Sep 26 09:16:30.624482 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: COMMIT
Sep 26 09:16:30.624955 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: # Completed by iptables_manager
Sep 26 09:16:30.635308 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent [None req-78fc6bc1-a089-4d5f-91d8-e5191e45978c None None] Error in agent loop. Devices info: {'current': set(['tapbc1a22b9-ef', 'tapc9488f0f-ae', 'tape2d2e245-96', 'tap93881b27-41', 'tapb265ee77-37', 'tapbadc6b64-69', 'tapa813220a-1d', 'tapa376782a-75', 'tap395ccf4d-c9', 'tapca94a412-e7', 'tap58f740f2-aa', 'tapb2444941-9f']), 'timestamps': {'tap93881b27-41': 56, 'tapc9488f0f-ae': 62, 'tape2d2e245-96': 11, 'tapbc1a22b9-ef': 68, 'tapb265ee77-37': 9, 'tapbadc6b64-69': 55, 'tapa813220a-1d': 66, 'tapa376782a-75': 65, 'tap395ccf4d-c9': 67, 'tapca94a412-e7': 6, 'tap58f740f2-aa': 59, 'tapb2444941-9f': 10}, 'removed': set([]), 'added': set([]), 'updated': set([])}: IpTablesApplyException: IPTables Rules did not converge. Diff: # Generated by iptables_manager
Sep 26 09:16:30.636316 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: *filter
Sep 26 09:16:30.636510 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -D neutron-linuxbri-ibc1a22b9-e 6
Sep 26 09:16:30.636700 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN
Sep 26 09:16:30.636898 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: COMMIT
Sep 26 09:16:30.637075 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: # Completed by iptables_manager
Sep 26 09:16:30.637269 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
Sep 26 09:16:30.637683 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 453, in daemon_loop
Sep 26 09:16:30.637962 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     sync = self.process_network_devices(device_info)
Sep 26 09:16:30.638211 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 157, in wrapper
Sep 26 09:16:30.638373 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     result = f(*args, **kwargs)
Sep 26 09:16:30.638538 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 200, in process_network_devices
Sep 26 09:16:30.638728 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     device_info.get('updated'))
Sep 26 09:16:30.639034 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 256, in setup_port_filters
Sep 26 09:16:30.639220 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.refresh_firewall(updated_devices)
Sep 26 09:16:30.639702 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 110, in decorated_function
Sep 26 09:16:30.639993 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     *args, **kwargs)
Sep 26 09:16:30.640390 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 209, in refresh_firewall
Sep 26 09:16:30.640671 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self._apply_port_filter(device_ids, update_filter=True)
Sep 26 09:16:30.640925 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 145, in _apply_port_filter
Sep 26 09:16:30.641159 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.firewall.process_trusted_ports(trusted_devices)
Sep 26 09:16:30.641466 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
Sep 26 09:16:30.641926 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.gen.next()
Sep 26 09:16:30.642208 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/firewall.py", line 145, in defer_apply
Sep 26 09:16:30.642454 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.filter_defer_apply_off()
Sep 26 09:16:30.642701 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 852, in filter_defer_apply_off
Sep 26 09:16:30.642990 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.iptables.defer_apply_off()
Sep 26 09:16:30.643349 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 429, in defer_apply_off
Sep 26 09:16:30.643647 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self._apply()
Sep 26 09:16:30.643920 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 454, in _apply
Sep 26 09:16:30.644170 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise n_exc.IpTablesApplyException(msg)
Sep 26 09:16:30.644519 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent IpTablesApplyException: IPTables Rules did not converge. Diff: # Generated by iptables_manager
Sep 26 09:16:30.644842 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent *filter
Sep 26 09:16:30.645208 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent -D neutron-linuxbri-ibc1a22b9-e 6
Sep 26 09:16:30.645479 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN
Sep 26 09:16:30.645847 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent COMMIT
Sep 26 09:16:30.646182 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent # Completed by iptables_manager
Sep 26 09:16:30.646457 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent 

This happens in scenario job when iptables manager is used. Despite the
fact that it doesn't happen in ovs flavor of the job, it's not clear
whether it affects ovs setups using iptables, because ovs scenario job
uses 'openvswitch' flow based firewall driver instead of iptables.

This happens on a patch that adds a new scenario test case targeting
security groups, so may be related:
https://review.openstack.org/#/c/504021/

** Affects: neutron
     Importance: High
         Status: Confirmed


** Tags: gate-failure linuxbridge sg-fw

** Changed in: neutron
   Importance: Undecided => High

** Changed in: neutron
       Status: New => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1719711

Title:
  iptables failed to apply when binding a port with
  AGENT.debug_iptables_rules enabled

Status in neutron:
  Confirmed

Bug description:
  http://logs.openstack.org/21/504021/2/check/gate-tempest-dsvm-neutron-
  scenario-linuxbridge-ubuntu-xenial-nv/e47a3f3/testr_results.html.gz

  
  Traceback (most recent call last):
    File "/opt/stack/new/neutron/neutron/tests/tempest/scenario/test_security_groups.py", line 127, in test_two_sec_groups
      num_servers=1, security_groups=security_groups_list)
    File "/opt/stack/new/neutron/neutron/tests/tempest/scenario/test_security_groups.py", line 54, in create_vm_testing_sec_grp
      const.SERVER_STATUS_ACTIVE)
    File "tempest/common/waiters.py", line 76, in wait_for_server_status
      server_id=server_id)
  tempest.exceptions.BuildErrorException: Server e1120d99-f0eb-43eb-a38b-847843a838b5 failed to build and is in ERROR status
  Details: {u'message': u'Build of instance e1120d99-f0eb-43eb-a38b-847843a838b5 aborted: Failed to allocate the network(s), not rescheduling.', u'code': 500, u'created': u'2017-09-26T09:23:42Z'}

  In linuxbridge agent log: http://logs.openstack.org/21/504021/2/check
  /gate-tempest-dsvm-neutron-scenario-linuxbridge-ubuntu-xenial-
  nv/e47a3f3/logs/screen-q-agt.txt.gz?level=TRACE#_Sep_26_09_16_30_623747

  Sep 26 09:16:30.623747 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.agent.linux.iptables_manager [None req-78fc6bc1-a089-4d5f-91d8-e5191e45978c None None] IPTables Rules did not converge. Diff: # Generated by iptables_manager
  Sep 26 09:16:30.623936 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: *filter
  Sep 26 09:16:30.624117 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -D neutron-linuxbri-ibc1a22b9-e 6
  Sep 26 09:16:30.624316 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN
  Sep 26 09:16:30.624482 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: COMMIT
  Sep 26 09:16:30.624955 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: # Completed by iptables_manager
  Sep 26 09:16:30.635308 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent [None req-78fc6bc1-a089-4d5f-91d8-e5191e45978c None None] Error in agent loop. Devices info: {'current': set(['tapbc1a22b9-ef', 'tapc9488f0f-ae', 'tape2d2e245-96', 'tap93881b27-41', 'tapb265ee77-37', 'tapbadc6b64-69', 'tapa813220a-1d', 'tapa376782a-75', 'tap395ccf4d-c9', 'tapca94a412-e7', 'tap58f740f2-aa', 'tapb2444941-9f']), 'timestamps': {'tap93881b27-41': 56, 'tapc9488f0f-ae': 62, 'tape2d2e245-96': 11, 'tapbc1a22b9-ef': 68, 'tapb265ee77-37': 9, 'tapbadc6b64-69': 55, 'tapa813220a-1d': 66, 'tapa376782a-75': 65, 'tap395ccf4d-c9': 67, 'tapca94a412-e7': 6, 'tap58f740f2-aa': 59, 'tapb2444941-9f': 10}, 'removed': set([]), 'added': set([]), 'updated': set([])}: IpTablesApplyException: IPTables Rules did not converge. Diff: # Generated by iptables_manager
  Sep 26 09:16:30.636316 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: *filter
  Sep 26 09:16:30.636510 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -D neutron-linuxbri-ibc1a22b9-e 6
  Sep 26 09:16:30.636700 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN
  Sep 26 09:16:30.636898 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: COMMIT
  Sep 26 09:16:30.637075 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: # Completed by iptables_manager
  Sep 26 09:16:30.637269 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):
  Sep 26 09:16:30.637683 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 453, in daemon_loop
  Sep 26 09:16:30.637962 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     sync = self.process_network_devices(device_info)
  Sep 26 09:16:30.638211 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 157, in wrapper
  Sep 26 09:16:30.638373 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     result = f(*args, **kwargs)
  Sep 26 09:16:30.638538 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 200, in process_network_devices
  Sep 26 09:16:30.638728 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     device_info.get('updated'))
  Sep 26 09:16:30.639034 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 256, in setup_port_filters
  Sep 26 09:16:30.639220 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.refresh_firewall(updated_devices)
  Sep 26 09:16:30.639702 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 110, in decorated_function
  Sep 26 09:16:30.639993 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     *args, **kwargs)
  Sep 26 09:16:30.640390 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 209, in refresh_firewall
  Sep 26 09:16:30.640671 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self._apply_port_filter(device_ids, update_filter=True)
  Sep 26 09:16:30.640925 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 145, in _apply_port_filter
  Sep 26 09:16:30.641159 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.firewall.process_trusted_ports(trusted_devices)
  Sep 26 09:16:30.641466 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  Sep 26 09:16:30.641926 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.gen.next()
  Sep 26 09:16:30.642208 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/firewall.py", line 145, in defer_apply
  Sep 26 09:16:30.642454 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.filter_defer_apply_off()
  Sep 26 09:16:30.642701 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 852, in filter_defer_apply_off
  Sep 26 09:16:30.642990 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.iptables.defer_apply_off()
  Sep 26 09:16:30.643349 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 429, in defer_apply_off
  Sep 26 09:16:30.643647 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     self._apply()
  Sep 26 09:16:30.643920 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 454, in _apply
  Sep 26 09:16:30.644170 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise n_exc.IpTablesApplyException(msg)
  Sep 26 09:16:30.644519 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent IpTablesApplyException: IPTables Rules did not converge. Diff: # Generated by iptables_manager
  Sep 26 09:16:30.644842 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent *filter
  Sep 26 09:16:30.645208 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent -D neutron-linuxbri-ibc1a22b9-e 6
  Sep 26 09:16:30.645479 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent -I neutron-linuxbri-ibc1a22b9-e 6 -p 1 -j RETURN
  Sep 26 09:16:30.645847 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent COMMIT
  Sep 26 09:16:30.646182 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent # Completed by iptables_manager
  Sep 26 09:16:30.646457 ubuntu-xenial-ovh-gra1-11134533 neutron-linuxbridge-agent[24363]: ERROR neutron.plugins.ml2.drivers.agent._common_agent 

  This happens in scenario job when iptables manager is used. Despite
  the fact that it doesn't happen in ovs flavor of the job, it's not
  clear whether it affects ovs setups using iptables, because ovs
  scenario job uses 'openvswitch' flow based firewall driver instead of
  iptables.

  This happens on a patch that adds a new scenario test case targeting
  security groups, so may be related:
  https://review.openstack.org/#/c/504021/

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1719711/+subscriptions
Follow ups

[Bug 1719711] Re: iptables failed to apply when binding a port with AGENT.debug_iptables_rules enabled
From: OpenStack Infra, 2017-12-09