yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1530042] [NEW] Missing iptables-ipv6 package cause ovs agent create flow failed

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Dongcan Ye <hellochosen@xxxxxxxxx>
Date: Wed, 30 Dec 2015 07:44:46 -0000
Reply-to: Bug 1530042 <1530042@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

In some scenario, Like OpenStack Kolla, system may lack iptables-ipv6 package.
This may  cause  command ip6tables-save or ip6tables-restore invalid. In this situation, when enabling security group, may cause ovs agent error:

2015-12-29 10:59:50.020 1 DEBUG neutron.agent.securitygroups_rpc [-] Preparing device filters for 1 new devices setup_port_filters /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:345
2015-12-29 10:59:50.021 1 INFO neutron.agent.securitygroups_rpc [-] Preparing filters for devices set([u'5735d7ab-6acf-4240-82a3-857da9ddeaf6'])
2015-12-29 10:59:50.022 1 DEBUG neutron.agent.securitygroups_rpc [-] Get security group information for devices via rpc [u'5735d7ab-6acf-4240-82a3-857da9ddeaf6'] security_group_info_for_devices /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:99
2015-12-29 10:59:50.023 1 DEBUG neutron.openstack.common.rpc.amqp [-] Making synchronous call on q-plugin ... multicall /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:566
2015-12-29 10:59:50.023 1 DEBUG neutron.openstack.common.rpc.amqp [-] MSG_ID is c49309ed837d4b9e8470314dece984e1 multicall /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:569
2015-12-29 10:59:50.024 1 DEBUG neutron.openstack.common.rpc.amqp [-] UNIQUE_ID is 0fd2207db4ab4f37831c913ce9ba4c18. _add_unique_id /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:350
2015-12-29 10:59:50.103 1 DEBUG neutron.agent.linux.iptables_firewall [-] Preparing device (5735d7ab-6acf-4240-82a3-857da9ddeaf6) filter prepare_port_filter /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py:75
2015-12-29 10:59:50.104 1 DEBUG neutron.agent.securitygroups_rpc [-] Update security group information for ports [u'5735d7ab-6acf-4240-82a3-857da9ddeaf6'] prepare_devices_filter /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:228
2015-12-29 10:59:50.105 1 DEBUG neutron.agent.securitygroups_rpc [-] Update security group information _update_security_group_info /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:234
2015-12-29 10:59:50.106 1 DEBUG neutron.agent.linux.iptables_firewall [-] Update rules of security group (1f21c2f6-8da1-4e90-af81-6ea9f0197c5f) update_security_group_rules /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py:67
2015-12-29 10:59:50.107 1 DEBUG neutron.agent.linux.iptables_firewall [-] Update members of security group (1f21c2f6-8da1-4e90-af81-6ea9f0197c5f) update_security_group_members /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py:71
2015-12-29 10:59:50.111 1 DEBUG neutron.openstack.common.lockutils [-] Got semaphore "iptables" lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:171
2015-12-29 10:59:50.112 1 DEBUG neutron.openstack.common.lockutils [-] Attempting to grab file lock "iptables" lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:181
2015-12-29 10:59:50.113 1 DEBUG neutron.openstack.common.lockutils [-] Got file lock "iptables" at /var/lib/neutron/lock/neutron-iptables lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:209
2015-12-29 10:59:50.114 1 DEBUG neutron.agent.linux.iptables_manager [-] Got semaphore / lock "iptables" _apply /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py:387
2015-12-29 10:59:50.116 1 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-save', '-c'] create_process /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:58
2015-12-29 10:59:50.289 1 DEBUG neutron.agent.linux.utils [-] 
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-save', '-c']
Exit code: 0
Stdout: '# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*raw\n:PREROUTING ACCEPT [119556:1153563515]\n:OUTPUT ACCEPT [106227:52170137]\n:neutron-openvswi-OUTPUT - [0:0]\n:neutron-openvswi-PREROUTING - [0:0]\n[31647:32857604] -A PREROUTING -j neutron-openvswi-PREROUTING \n[0:0] -A PREROUTING -p gre -m comment --comment "333 notrack gre" -j NOTRACK \n[39061:45136621] -A OUTPUT -j neutron-openvswi-OUTPUT \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*nat\n:PREROUTING ACCEPT [1940:78467]\n:POSTROUTING ACCEPT [108:7227]\n:OUTPUT ACCEPT [105:6300]\n:DOCKER - [0:0]\n:neutron-openvswi-OUTPUT - [0:0]\n:neutron-openvswi-POSTROUTING - [0:0]\n:neutron-openvswi-PREROUTING - [0:0]\n:neutron-openvswi-float-snat - [0:0]\n:neutron-openvswi-snat - [0:0]\n:neutron-postrouting-bottom - [0:0]\n[395:16647] -A PREROUTING -j neutron-openvswi-PREROUTING \n[73:4380] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER \n[36:2907] -A POSTROUTING -j neutron-openvswi-POSTROUTING \n[36:2907] -A POSTROUTING -j neutron-postrouting-bottom \n[0:0] -A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN \n[0:0] -A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN \n[0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 \n[0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 \n[0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE \n[0:0] -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE \n[33:1980] -A OUTPUT -j neutron-openvswi-OUTPUT \n[0:0] -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER \n[36:2907] -A neutron-openvswi-snat -j neutron-openvswi-float-snat \n[36:2907] -A neutron-postrouting-bottom -j neutron-openvswi-snat \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*mangle\n:PREROUTING ACCEPT [68870:358570045]\n:INPUT ACCEPT [65002:358415325]\n:FORWARD ACCEPT [1937:78287]\n:OUTPUT ACCEPT [69746:48620665]\n:POSTROUTING ACCEPT [71683:48698952]\n:neutron-openvswi-POSTROUTING - [0:0]\n[39078:44624498] -A POSTROUTING -j neutron-openvswi-POSTROUTING \n[0:0] -A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill \n[4:745] -A POSTROUTING -p udp -m udp --dport 514 -j CHECKSUM --checksum-fill \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*filter\n:INPUT ACCEPT [0:0]\n:FORWARD ACCEPT [1934:77360]\n:OUTPUT ACCEPT [69061:47596876]\n:DOCKER - [0:0]\n:neutron-filter-top - [0:0]\n:neutron-openvswi-FORWARD - [0:0]\n:neutron-openvswi-INPUT - [0:0]\n:neutron-openvswi-OUTPUT - [0:0]\n:neutron-openvswi-i5735d7ab-6 - [0:0]\n:neutron-openvswi-local - [0:0]\n:neutron-openvswi-o5735d7ab-6 - [0:0]\n:neutron-openvswi-s5735d7ab-6 - [0:0]\n:neutron-openvswi-sg-chain - [0:0]\n:neutron-openvswi-sg-fallback - [0:0]\n[30114:32143705] -A INPUT -j neutron-openvswi-INPUT \n[0:0] -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT \n[0:0] -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT \n[0:0] -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT \n[0:0] -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT \n[0:0] -A INPUT -p icmp -m comment --comment "000 accept all icmp requests" -j ACCEPT \n[4211:221092] -A INPUT -i lo -m comment --comment "001 accept all to lo interface" -j ACCEPT \n[109877:1152500281] -A INPUT -m comment --comment "002 accept related established rules" -m state --state RELATED,ESTABLISHED -j ACCEPT \n[0:0] -A INPUT -s 10.20.0.2/32 -p tcp -m multiport --sports 4369,5672,15672,41055,55672,61613 -m comment --comment "003 remote rabbitmq " -j ACCEPT \n[0:0] -A INPUT -s 10.20.0.2/32 -p tcp -m multiport --sports 8140 -m comment --comment "004 remote puppet " -j ACCEPT \n[73:4380] -A INPUT -p tcp -m multiport --ports 22 -m comment --comment "020 ssh" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 80,443 -m comment --comment "100 http" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 3306,3307,4567,4568,4444,49000 -m comment --comment "101 mysql" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 5000,35357 -m comment --comment "102 keystone" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 8080,6000,6001,6002,49001 -m comment --comment "103 swift" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 9292,9191,8773 -m comment --comment "104 glance" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 8774,8775,8776,6080 -m comment --comment "105 nova " -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 4369,5672,5673,41055 -m comment --comment "106 rabbitmq " -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 11211 -m comment --comment "107 memcached tcp" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 11211 -m comment --comment "107 memcached udp" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 873 -m comment --comment "108 rsync" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 3260 -m comment --comment "109 iscsi " -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 9696 -m comment --comment "110 neutron " -j ACCEPT \n[3:927] -A INPUT -p udp -m multiport --ports 67 -m comment --comment "111 dhcp-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 53 -m comment --comment "111 dns-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 123 -m comment --comment "112 ntp-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 5404 -m comment --comment "113 corosync-input" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 5405 -m comment --comment "114 corosync-output" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 2224 -m comment --comment "115 pcsd-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 58882 -m comment --comment "116 openvswitch db" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 5666 -m comment --comment "117 nrpe-server" -j ACCEPT \n[0:0] -A INPUT -s 192.168.0.0/24 -p tcp -m multiport --ports 16509 -m comment --comment "118 libvirt" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 49152:49215 -m comment --comment "119 libvirt migration" -j ACCEPT \n[0:0] -A INPUT -s 192.168.0.0/24 -p tcp -m multiport --ports 5900:6100 -m comment --comment "120 vnc ports" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 8777 -m comment --comment "121 ceilometer" -j ACCEPT \n[0:0] -A INPUT -p gre -m comment --comment "334 accept gre" -j ACCEPT \n[2:80] -A INPUT -m comment --comment "999 drop all other requests" -j DROP \n[393:16527] -A FORWARD -j neutron-filter-top \n[393:16527] -A FORWARD -j neutron-openvswi-FORWARD \n[0:0] -A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT \n[0:0] -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT \n[0:0] -A FORWARD -i virbr0 -o virbr0 -j ACCEPT \n[0:0] -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable \n[0:0] -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable \n[0:0] -A FORWARD -o docker0 -j DOCKER \n[0:0] -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT \n[0:0] -A FORWARD -i docker0 ! -o docker0 -j ACCEPT \n[0:0] -A FORWARD -i docker0 -o docker0 -j ACCEPT \n[37916:43579541] -A OUTPUT -j neutron-filter-top \n[37916:43579541] -A OUTPUT -j neutron-openvswi-OUTPUT \n[0:0] -A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT \n[38309:43596068] -A neutron-filter-top -j neutron-openvswi-local \n[0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-sg-chain \n[3:927] -A neutron-openvswi-FORWARD -m physdev --physdev-in tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-sg-chain \n[0:0] -A neutron-openvswi-INPUT -m physdev --physdev-in tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-o5735d7ab-6 \n[0:0] -A neutron-openvswi-i5735d7ab-6 -m state --state INVALID -j DROP \n[0:0] -A neutron-openvswi-i5735d7ab-6 -m state --state RELATED,ESTABLISHED -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -s 192.168.111.3/32 -p udp -m udp --sport 67 --dport 68 -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -p icmp -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -s 192.168.111.4/32 -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -j neutron-openvswi-sg-fallback \n[3:927] -A neutron-openvswi-o5735d7ab-6 -p udp -m udp --sport 68 --dport 67 -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -j neutron-openvswi-s5735d7ab-6 \n[0:0] -A neutron-openvswi-o5735d7ab-6 -p udp -m udp --sport 67 --dport 68 -j DROP \n[0:0] -A neutron-openvswi-o5735d7ab-6 -m state --state INVALID -j DROP \n[0:0] -A neutron-openvswi-o5735d7ab-6 -m state --state RELATED,ESTABLISHED -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -p icmp -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -j neutron-openvswi-sg-fallback \n[0:0] -A neutron-openvswi-s5735d7ab-6 -s 192.168.111.2/32 -m mac --mac-source FA:16:3E:DE:22:11 -j RETURN \n[0:0] -A neutron-openvswi-s5735d7ab-6 -j DROP \n[0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-out tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-i5735d7ab-6 \n[3:927] -A neutron-openvswi-sg-chain -m physdev --physdev-in tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-o5735d7ab-6 \n[3:927] -A neutron-openvswi-sg-chain -j ACCEPT \n[0:0] -A neutron-openvswi-sg-fallback -j DROP \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n'
Stderr: '' _execute /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:94
2015-12-29 10:59:50.298 1 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-restore', '-c'] create_process /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:58
2015-12-29 10:59:50.468 1 DEBUG neutron.agent.linux.utils [-] 
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-restore', '-c']
Exit code: 0
Stdout: ''
Stderr: '' _execute /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:94
2015-12-29 10:59:50.470 1 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-save', '-c'] create_process /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:58
2015-12-29 10:59:50.604 1 ERROR neutron.agent.linux.utils [-] 
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-save', '-c']
Exit code: 96
Stdout: ''
Stderr: '/usr/bin/neutron-rootwrap: Executable not found: ip6tables-save (filter match = ip6tables-save)\n'
2015-12-29 10:59:50.605 1 DEBUG neutron.openstack.common.lockutils [-] Released file lock "iptables" at /var/lib/neutron/lock/neutron-iptables lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:213
2015-12-29 10:59:50.606 1 DEBUG neutron.agent.linux.iptables_manager [-] Semaphore / lock released "iptables" _apply /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py:390
2015-12-29 10:59:50.607 1 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Error while processing VIF ports
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1299, in rpc_loop
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     ovs_restarted)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1126, in process_network_ports
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     port_info.get('updated', set()))
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 346, in setup_port_filters
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.prepare_devices_filter(new_devices)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 205, in decorated_function
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     return func(self, *args, **kwargs)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 230, in prepare_devices_filter
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     security_groups, security_group_member_ips)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.gen.next()
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/firewall.py", line 109, in defer_apply
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py", line 436, in filter_defer_apply_off
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 372, in defer_apply_off
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self._apply()
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 388, in _apply
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     return self._apply_synchronized()
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 408, in _apply_synchronized
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     all_tables = self.execute(args, root_helper=self.root_helper)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 118, in execute
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     extra_ok_codes)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 92, in _execute
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(m)
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent RuntimeError: 
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-save', '-c']
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Exit code: 96
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdout: ''
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: '/usr/bin/neutron-rootwrap: Executable not found: ip6tables-save (filter match = ip6tables-save)\n'
2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent 
2015-12-29 10:59:50.611 1 DEBUG neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent rpc_loop - iteration:1005 completed. Processed ports statistics: {'ancillary': {'removed': 0, 'added': 0}, 'regular': {'updated': 0, 'added': 0, 'removed': 0}}. Elapsed:1.472 rpc_loop /usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1349
2015-12-29 10:59:51.139 1 DEBUG neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent rpc_loop - iteration:1006 started rpc_loop /usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1248

This problem can avoid by using neutron sanity check tool.

** Affects: neutron
     Importance: Undecided
     Assignee: Dongcan Ye (hellochosen)
         Status: New

** Changed in: neutron
     Assignee: (unassigned) => Dongcan Ye (hellochosen)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1530042

Title:
  Missing iptables-ipv6 package cause ovs agent create flow failed

Status in neutron:
  New

Bug description:
  In some scenario, Like OpenStack Kolla, system may lack iptables-ipv6 package.
  This may  cause  command ip6tables-save or ip6tables-restore invalid. In this situation, when enabling security group, may cause ovs agent error:

  2015-12-29 10:59:50.020 1 DEBUG neutron.agent.securitygroups_rpc [-] Preparing device filters for 1 new devices setup_port_filters /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:345
  2015-12-29 10:59:50.021 1 INFO neutron.agent.securitygroups_rpc [-] Preparing filters for devices set([u'5735d7ab-6acf-4240-82a3-857da9ddeaf6'])
  2015-12-29 10:59:50.022 1 DEBUG neutron.agent.securitygroups_rpc [-] Get security group information for devices via rpc [u'5735d7ab-6acf-4240-82a3-857da9ddeaf6'] security_group_info_for_devices /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:99
  2015-12-29 10:59:50.023 1 DEBUG neutron.openstack.common.rpc.amqp [-] Making synchronous call on q-plugin ... multicall /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:566
  2015-12-29 10:59:50.023 1 DEBUG neutron.openstack.common.rpc.amqp [-] MSG_ID is c49309ed837d4b9e8470314dece984e1 multicall /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:569
  2015-12-29 10:59:50.024 1 DEBUG neutron.openstack.common.rpc.amqp [-] UNIQUE_ID is 0fd2207db4ab4f37831c913ce9ba4c18. _add_unique_id /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:350
  2015-12-29 10:59:50.103 1 DEBUG neutron.agent.linux.iptables_firewall [-] Preparing device (5735d7ab-6acf-4240-82a3-857da9ddeaf6) filter prepare_port_filter /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py:75
  2015-12-29 10:59:50.104 1 DEBUG neutron.agent.securitygroups_rpc [-] Update security group information for ports [u'5735d7ab-6acf-4240-82a3-857da9ddeaf6'] prepare_devices_filter /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:228
  2015-12-29 10:59:50.105 1 DEBUG neutron.agent.securitygroups_rpc [-] Update security group information _update_security_group_info /usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py:234
  2015-12-29 10:59:50.106 1 DEBUG neutron.agent.linux.iptables_firewall [-] Update rules of security group (1f21c2f6-8da1-4e90-af81-6ea9f0197c5f) update_security_group_rules /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py:67
  2015-12-29 10:59:50.107 1 DEBUG neutron.agent.linux.iptables_firewall [-] Update members of security group (1f21c2f6-8da1-4e90-af81-6ea9f0197c5f) update_security_group_members /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py:71
  2015-12-29 10:59:50.111 1 DEBUG neutron.openstack.common.lockutils [-] Got semaphore "iptables" lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:171
  2015-12-29 10:59:50.112 1 DEBUG neutron.openstack.common.lockutils [-] Attempting to grab file lock "iptables" lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:181
  2015-12-29 10:59:50.113 1 DEBUG neutron.openstack.common.lockutils [-] Got file lock "iptables" at /var/lib/neutron/lock/neutron-iptables lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:209
  2015-12-29 10:59:50.114 1 DEBUG neutron.agent.linux.iptables_manager [-] Got semaphore / lock "iptables" _apply /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py:387
  2015-12-29 10:59:50.116 1 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-save', '-c'] create_process /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:58
  2015-12-29 10:59:50.289 1 DEBUG neutron.agent.linux.utils [-] 
  Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-save', '-c']
  Exit code: 0
  Stdout: '# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*raw\n:PREROUTING ACCEPT [119556:1153563515]\n:OUTPUT ACCEPT [106227:52170137]\n:neutron-openvswi-OUTPUT - [0:0]\n:neutron-openvswi-PREROUTING - [0:0]\n[31647:32857604] -A PREROUTING -j neutron-openvswi-PREROUTING \n[0:0] -A PREROUTING -p gre -m comment --comment "333 notrack gre" -j NOTRACK \n[39061:45136621] -A OUTPUT -j neutron-openvswi-OUTPUT \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*nat\n:PREROUTING ACCEPT [1940:78467]\n:POSTROUTING ACCEPT [108:7227]\n:OUTPUT ACCEPT [105:6300]\n:DOCKER - [0:0]\n:neutron-openvswi-OUTPUT - [0:0]\n:neutron-openvswi-POSTROUTING - [0:0]\n:neutron-openvswi-PREROUTING - [0:0]\n:neutron-openvswi-float-snat - [0:0]\n:neutron-openvswi-snat - [0:0]\n:neutron-postrouting-bottom - [0:0]\n[395:16647] -A PREROUTING -j neutron-openvswi-PREROUTING \n[73:4380] -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER \n[36:2907] -A POSTROUTING -j neutron-openvswi-POSTROUTING \n[36:2907] -A POSTROUTING -j neutron-postrouting-bottom \n[0:0] -A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN \n[0:0] -A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN \n[0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535 \n[0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535 \n[0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE \n[0:0] -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE \n[33:1980] -A OUTPUT -j neutron-openvswi-OUTPUT \n[0:0] -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER \n[36:2907] -A neutron-openvswi-snat -j neutron-openvswi-float-snat \n[36:2907] -A neutron-postrouting-bottom -j neutron-openvswi-snat \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*mangle\n:PREROUTING ACCEPT [68870:358570045]\n:INPUT ACCEPT [65002:358415325]\n:FORWARD ACCEPT [1937:78287]\n:OUTPUT ACCEPT [69746:48620665]\n:POSTROUTING ACCEPT [71683:48698952]\n:neutron-openvswi-POSTROUTING - [0:0]\n[39078:44624498] -A POSTROUTING -j neutron-openvswi-POSTROUTING \n[0:0] -A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill \n[4:745] -A POSTROUTING -p udp -m udp --dport 514 -j CHECKSUM --checksum-fill \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n# Generated by iptables-save v1.4.7 on Tue Dec 29 10:59:50 2015\n*filter\n:INPUT ACCEPT [0:0]\n:FORWARD ACCEPT [1934:77360]\n:OUTPUT ACCEPT [69061:47596876]\n:DOCKER - [0:0]\n:neutron-filter-top - [0:0]\n:neutron-openvswi-FORWARD - [0:0]\n:neutron-openvswi-INPUT - [0:0]\n:neutron-openvswi-OUTPUT - [0:0]\n:neutron-openvswi-i5735d7ab-6 - [0:0]\n:neutron-openvswi-local - [0:0]\n:neutron-openvswi-o5735d7ab-6 - [0:0]\n:neutron-openvswi-s5735d7ab-6 - [0:0]\n:neutron-openvswi-sg-chain - [0:0]\n:neutron-openvswi-sg-fallback - [0:0]\n[30114:32143705] -A INPUT -j neutron-openvswi-INPUT \n[0:0] -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT \n[0:0] -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT \n[0:0] -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT \n[0:0] -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT \n[0:0] -A INPUT -p icmp -m comment --comment "000 accept all icmp requests" -j ACCEPT \n[4211:221092] -A INPUT -i lo -m comment --comment "001 accept all to lo interface" -j ACCEPT \n[109877:1152500281] -A INPUT -m comment --comment "002 accept related established rules" -m state --state RELATED,ESTABLISHED -j ACCEPT \n[0:0] -A INPUT -s 10.20.0.2/32 -p tcp -m multiport --sports 4369,5672,15672,41055,55672,61613 -m comment --comment "003 remote rabbitmq " -j ACCEPT \n[0:0] -A INPUT -s 10.20.0.2/32 -p tcp -m multiport --sports 8140 -m comment --comment "004 remote puppet " -j ACCEPT \n[73:4380] -A INPUT -p tcp -m multiport --ports 22 -m comment --comment "020 ssh" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 80,443 -m comment --comment "100 http" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 3306,3307,4567,4568,4444,49000 -m comment --comment "101 mysql" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 5000,35357 -m comment --comment "102 keystone" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 8080,6000,6001,6002,49001 -m comment --comment "103 swift" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 9292,9191,8773 -m comment --comment "104 glance" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 8774,8775,8776,6080 -m comment --comment "105 nova " -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 4369,5672,5673,41055 -m comment --comment "106 rabbitmq " -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 11211 -m comment --comment "107 memcached tcp" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 11211 -m comment --comment "107 memcached udp" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 873 -m comment --comment "108 rsync" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 3260 -m comment --comment "109 iscsi " -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 9696 -m comment --comment "110 neutron " -j ACCEPT \n[3:927] -A INPUT -p udp -m multiport --ports 67 -m comment --comment "111 dhcp-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 53 -m comment --comment "111 dns-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 123 -m comment --comment "112 ntp-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 5404 -m comment --comment "113 corosync-input" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 5405 -m comment --comment "114 corosync-output" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 2224 -m comment --comment "115 pcsd-server" -j ACCEPT \n[0:0] -A INPUT -p udp -m multiport --ports 58882 -m comment --comment "116 openvswitch db" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 5666 -m comment --comment "117 nrpe-server" -j ACCEPT \n[0:0] -A INPUT -s 192.168.0.0/24 -p tcp -m multiport --ports 16509 -m comment --comment "118 libvirt" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 49152:49215 -m comment --comment "119 libvirt migration" -j ACCEPT \n[0:0] -A INPUT -s 192.168.0.0/24 -p tcp -m multiport --ports 5900:6100 -m comment --comment "120 vnc ports" -j ACCEPT \n[0:0] -A INPUT -p tcp -m multiport --ports 8777 -m comment --comment "121 ceilometer" -j ACCEPT \n[0:0] -A INPUT -p gre -m comment --comment "334 accept gre" -j ACCEPT \n[2:80] -A INPUT -m comment --comment "999 drop all other requests" -j DROP \n[393:16527] -A FORWARD -j neutron-filter-top \n[393:16527] -A FORWARD -j neutron-openvswi-FORWARD \n[0:0] -A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT \n[0:0] -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT \n[0:0] -A FORWARD -i virbr0 -o virbr0 -j ACCEPT \n[0:0] -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable \n[0:0] -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable \n[0:0] -A FORWARD -o docker0 -j DOCKER \n[0:0] -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT \n[0:0] -A FORWARD -i docker0 ! -o docker0 -j ACCEPT \n[0:0] -A FORWARD -i docker0 -o docker0 -j ACCEPT \n[37916:43579541] -A OUTPUT -j neutron-filter-top \n[37916:43579541] -A OUTPUT -j neutron-openvswi-OUTPUT \n[0:0] -A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT \n[38309:43596068] -A neutron-filter-top -j neutron-openvswi-local \n[0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-sg-chain \n[3:927] -A neutron-openvswi-FORWARD -m physdev --physdev-in tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-sg-chain \n[0:0] -A neutron-openvswi-INPUT -m physdev --physdev-in tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-o5735d7ab-6 \n[0:0] -A neutron-openvswi-i5735d7ab-6 -m state --state INVALID -j DROP \n[0:0] -A neutron-openvswi-i5735d7ab-6 -m state --state RELATED,ESTABLISHED -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -s 192.168.111.3/32 -p udp -m udp --sport 67 --dport 68 -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -p icmp -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -s 192.168.111.4/32 -j RETURN \n[0:0] -A neutron-openvswi-i5735d7ab-6 -j neutron-openvswi-sg-fallback \n[3:927] -A neutron-openvswi-o5735d7ab-6 -p udp -m udp --sport 68 --dport 67 -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -j neutron-openvswi-s5735d7ab-6 \n[0:0] -A neutron-openvswi-o5735d7ab-6 -p udp -m udp --sport 67 --dport 68 -j DROP \n[0:0] -A neutron-openvswi-o5735d7ab-6 -m state --state INVALID -j DROP \n[0:0] -A neutron-openvswi-o5735d7ab-6 -m state --state RELATED,ESTABLISHED -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -p icmp -j RETURN \n[0:0] -A neutron-openvswi-o5735d7ab-6 -j neutron-openvswi-sg-fallback \n[0:0] -A neutron-openvswi-s5735d7ab-6 -s 192.168.111.2/32 -m mac --mac-source FA:16:3E:DE:22:11 -j RETURN \n[0:0] -A neutron-openvswi-s5735d7ab-6 -j DROP \n[0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-out tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-i5735d7ab-6 \n[3:927] -A neutron-openvswi-sg-chain -m physdev --physdev-in tap5735d7ab-6a --physdev-is-bridged -j neutron-openvswi-o5735d7ab-6 \n[3:927] -A neutron-openvswi-sg-chain -j ACCEPT \n[0:0] -A neutron-openvswi-sg-fallback -j DROP \nCOMMIT\n# Completed on Tue Dec 29 10:59:50 2015\n'
  Stderr: '' _execute /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:94
  2015-12-29 10:59:50.298 1 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-restore', '-c'] create_process /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:58
  2015-12-29 10:59:50.468 1 DEBUG neutron.agent.linux.utils [-] 
  Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-restore', '-c']
  Exit code: 0
  Stdout: ''
  Stderr: '' _execute /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:94
  2015-12-29 10:59:50.470 1 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-save', '-c'] create_process /usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py:58
  2015-12-29 10:59:50.604 1 ERROR neutron.agent.linux.utils [-] 
  Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-save', '-c']
  Exit code: 96
  Stdout: ''
  Stderr: '/usr/bin/neutron-rootwrap: Executable not found: ip6tables-save (filter match = ip6tables-save)\n'
  2015-12-29 10:59:50.605 1 DEBUG neutron.openstack.common.lockutils [-] Released file lock "iptables" at /var/lib/neutron/lock/neutron-iptables lock /usr/lib/python2.6/site-packages/neutron/openstack/common/lockutils.py:213
  2015-12-29 10:59:50.606 1 DEBUG neutron.agent.linux.iptables_manager [-] Semaphore / lock released "iptables" _apply /usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py:390
  2015-12-29 10:59:50.607 1 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Error while processing VIF ports
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1299, in rpc_loop
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     ovs_restarted)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1126, in process_network_ports
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     port_info.get('updated', set()))
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 346, in setup_port_filters
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.prepare_devices_filter(new_devices)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 205, in decorated_function
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     return func(self, *args, **kwargs)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/securitygroups_rpc.py", line 230, in prepare_devices_filter
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     security_groups, security_group_member_ips)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.gen.next()
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/firewall.py", line 109, in defer_apply
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_firewall.py", line 436, in filter_defer_apply_off
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 372, in defer_apply_off
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self._apply()
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 388, in _apply
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     return self._apply_synchronized()
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/iptables_manager.py", line 408, in _apply_synchronized
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     all_tables = self.execute(args, root_helper=self.root_helper)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 118, in execute
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     extra_ok_codes)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 92, in _execute
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(m)
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent RuntimeError: 
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip6tables-save', '-c']
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Exit code: 96
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdout: ''
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: '/usr/bin/neutron-rootwrap: Executable not found: ip6tables-save (filter match = ip6tables-save)\n'
  2015-12-29 10:59:50.607 1 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent 
  2015-12-29 10:59:50.611 1 DEBUG neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent rpc_loop - iteration:1005 completed. Processed ports statistics: {'ancillary': {'removed': 0, 'added': 0}, 'regular': {'updated': 0, 'added': 0, 'removed': 0}}. Elapsed:1.472 rpc_loop /usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1349
  2015-12-29 10:59:51.139 1 DEBUG neutron.plugins.openvswitch.agent.ovs_neutron_agent [-] Agent rpc_loop - iteration:1006 started rpc_loop /usr/lib/python2.6/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py:1248

  This problem can avoid by using neutron sanity check tool.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1530042/+subscriptions
Follow ups

[Bug 1530042] Re: Missing iptables-ipv6 package cause ovs agent create flow failed
From: OpenStack Infra, 2016-01-08