yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1575033] [NEW] iptables-restore fails with RuntimeError for ipset

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: venkata anil <anilvenkata@xxxxxxxxxx>
Date: Tue, 26 Apr 2016 08:20:58 -0000
Reply-to: Bug 1575033 <1575033@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

The following Trace is seen ovs_neutron_agent while running functional
tests

http://logs.openstack.org/59/307159/5/check/gate-neutron-dsvm-
fullstack/e1f25d4/logs/TestDVRL3Agent.test_snat_and_floatingip/neutron-
openvswitch-agent--2016-04-22--12-33-51-032511.log.txt.gz


2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-f775b822-c40d-4d94-a2ec-005fb8b038fb - - - - -] Error while processing VIF ports
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1992, in rpc_loop
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info, ovs_restarted)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1623, in process_network_ports
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info.get('updated', set()))
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 292, in setup_port_filters
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.prepare_devices_filter(new_devices)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 147, in decorated_function
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     *args, **kwargs)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 172, in prepare_devices_filter
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.firewall.prepare_port_filter(device)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.gen.next()
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/firewall.py", line 129, in defer_apply
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 824, in filter_defer_apply_off
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 824, in filter_defer_apply_off
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 468, in defer_apply_off
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self._apply()
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 482, in _apply
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     return self._apply_synchronized()
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 559, in _apply_synchronized
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     '\n'.join(log_lines))
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/.tox/dsvm-fullstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.force_reraise()
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/.tox/dsvm-fullstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     six.reraise(self.type_, self.value, self.tb)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 538, in _apply_synchronized
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     run_as_root=True)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 137, in execute
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(msg)
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent RuntimeError: Exit code: 2; Stdin: # Generated by iptables_manager
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent *filter
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent :ovs_agent.py-ib272c437-0 - [0:0]

2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent -I ovs_agent.py-PREROUTING 2 -m physdev --physdev-in tapb272c437-05 -j CT --zone 1
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent COMMIT
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent # Completed by iptables_manager
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent ; Stdout: ; Stderr: iptables-restore v1.4.21: Set NIPv42a80b4e9-6d6f-4847-9e62- doesn't exist.
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Error occurred at line: 10
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Try `iptables-restore -h' or 'iptables-restore --help' for more information.
2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent

** Affects: neutron
     Importance: Undecided
     Assignee: venkata anil (anil-venkata)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => venkata anil (anil-venkata)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1575033

Title:
  iptables-restore fails with RuntimeError for ipset

Status in neutron:
  New

Bug description:
  The following Trace is seen ovs_neutron_agent while running functional
  tests

  http://logs.openstack.org/59/307159/5/check/gate-neutron-dsvm-
  fullstack/e1f25d4/logs/TestDVRL3Agent.test_snat_and_floatingip
  /neutron-openvswitch-agent--2016-04-22--12-33-51-032511.log.txt.gz


  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-f775b822-c40d-4d94-a2ec-005fb8b038fb - - - - -] Error while processing VIF ports
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1992, in rpc_loop
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info, ovs_restarted)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py", line 1623, in process_network_ports
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     port_info.get('updated', set()))
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 292, in setup_port_filters
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.prepare_devices_filter(new_devices)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 147, in decorated_function
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     *args, **kwargs)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/securitygroups_rpc.py", line 172, in prepare_devices_filter
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.firewall.prepare_port_filter(device)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.gen.next()
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/firewall.py", line 129, in defer_apply
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 824, in filter_defer_apply_off
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_firewall.py", line 824, in filter_defer_apply_off
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 468, in defer_apply_off
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self._apply()
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 482, in _apply
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     return self._apply_synchronized()
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 559, in _apply_synchronized
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     '\n'.join(log_lines))
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/.tox/dsvm-fullstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     self.force_reraise()
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/.tox/dsvm-fullstack/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     six.reraise(self.type_, self.value, self.tb)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/iptables_manager.py", line 538, in _apply_synchronized
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     run_as_root=True)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent   File "/opt/stack/new/neutron/neutron/agent/linux/utils.py", line 137, in execute
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(msg)
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent RuntimeError: Exit code: 2; Stdin: # Generated by iptables_manager
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent *filter
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent :ovs_agent.py-ib272c437-0 - [0:0]

  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent -I ovs_agent.py-PREROUTING 2 -m physdev --physdev-in tapb272c437-05 -j CT --zone 1
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent COMMIT
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent # Completed by iptables_manager
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent ; Stdout: ; Stderr: iptables-restore v1.4.21: Set NIPv42a80b4e9-6d6f-4847-9e62- doesn't exist.
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Error occurred at line: 10
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Try `iptables-restore -h' or 'iptables-restore --help' for more information.
  2016-04-22 12:34:33.670 27936 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1575033/+subscriptions
Follow ups

[Bug 1575033] Re: [Fullstack] iptables-restore fails with RuntimeError for ipset
From: OpenStack Infra, 2016-05-02