yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1379779] Re: neutron-openvswitch-agent fails to apply iptables rules

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: James Page <james.page@xxxxxxxxxx>
Date: Fri, 10 Oct 2014 13:00:24 -0000
Reply-to: Bug 1379779 <1379779@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
1. # Generated by iptables-save v1.4.21 on Fri Oct 10 12:57:46 2014
      2. *raw
      3. :PREROUTING ACCEPT [14112:2558828]
      4. :OUTPUT ACCEPT [15144:2771232]
      5. :neutron-openvswi-OUTPUT - [0:0]
      6. :neutron-openvswi-PREROUTING - [0:0]
      7. [14112:2558828] -A PREROUTING -j neutron-openvswi-PREROUTING
      8. [15144:2771232] -A OUTPUT -j neutron-openvswi-OUTPUT
      9. COMMIT
     10. # Completed on Fri Oct 10 12:57:46 2014
     11. # Generated by iptables-save v1.4.21 on Fri Oct 10 12:57:46 2014
     12. *mangle
     13. :PREROUTING ACCEPT [32301:28693852]
     14. :INPUT ACCEPT [32291:28693414]
     15. :FORWARD ACCEPT [0:0]
     16. :OUTPUT ACCEPT [28668:5226155]
     17. :POSTROUTING ACCEPT [28668:5226155]
     18. [0:0] -A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
     19. COMMIT
     20. # Completed on Fri Oct 10 12:57:46 2014
     21. # Generated by iptables-save v1.4.21 on Fri Oct 10 12:57:46 2014
     22. *nat
     23. :PREROUTING ACCEPT [11:498]
     24. :INPUT ACCEPT [1:60]
     25. :OUTPUT ACCEPT [3960:318233]
     26. :POSTROUTING ACCEPT [3960:318233]
     27. :neutron-postrouting-bottom - [0:0]
     28. :neutron-openvswi-OUTPUT - [0:0]
     29. :neutron-openvswi-POSTROUTING - [0:0]
     30. :neutron-openvswi-PREROUTING - [0:0]
     31. :neutron-openvswi-float-snat - [0:0]
     32. :neutron-openvswi-snat - [0:0]
     33. [3:140] -A PREROUTING -j neutron-openvswi-PREROUTING
     34. [2312:186295] -A OUTPUT -j neutron-openvswi-OUTPUT
     35. [2312:186295] -A POSTROUTING -j neutron-openvswi-POSTROUTING
     36. [2312:186295] -A POSTROUTING -j neutron-postrouting-bottom
     37. [2312:186295] -A neutron-postrouting-bottom -j neutron-openvswi-snat
     38. [2312:186295] -A neutron-openvswi-snat -j neutron-openvswi-float-snat
     39. [0:0] -A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
     40. [0:0] -A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
     41. [0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
     42. [0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
     43. [0:0] -A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
     44. COMMIT
     45. # Completed on Fri Oct 10 12:57:46 2014
     46. # Generated by iptables-save v1.4.21 on Fri Oct 10 12:57:46 2014
     47. *filter
     48. :INPUT ACCEPT [32961:28761138]
     49. :FORWARD ACCEPT [0:0]
     50. :OUTPUT ACCEPT [29341:5283975]
     51. :neutron-filter-top - [0:0]
     52. :neutron-openvswi-FORWARD - [0:0]
     53. :neutron-openvswi-INPUT - [0:0]
     54. :neutron-openvswi-OUTPUT - [0:0]
     55. :neutron-openvswi-i3d3f7a31-9 - [0:0]
     56. :neutron-openvswi-i62de4e08-b - [0:0]
     57. :neutron-openvswi-i7010a0ba-c - [0:0]
     58. :neutron-openvswi-local - [0:0]
     59. :neutron-openvswi-o3d3f7a31-9 - [0:0]
     60. :neutron-openvswi-o62de4e08-b - [0:0]
     61. :neutron-openvswi-o7010a0ba-c - [0:0]
     62. :neutron-openvswi-s3d3f7a31-9 - [0:0]
     63. :neutron-openvswi-s62de4e08-b - [0:0]
     64. :neutron-openvswi-s7010a0ba-c - [0:0]
     65. :neutron-openvswi-sg-chain - [0:0]
     66. :neutron-openvswi-sg-fallback - [0:0]
     67. [0:0] -A FORWARD -j neutron-filter-top
     68. [0:0] -A OUTPUT -j neutron-filter-top
     69. [0:0] -A neutron-filter-top -j neutron-openvswi-local
     70. [0:0] -A INPUT -j neutron-openvswi-INPUT
     71. [0:0] -A OUTPUT -j neutron-openvswi-OUTPUT
     72. [0:0] -A FORWARD -j neutron-openvswi-FORWARD
     73. [0:0] -A neutron-openvswi-sg-fallback -j DROP
     74. [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap62de4e08-b8 --physdev-is-bridged -j neutron-openvswi-sg-chain
     75. [0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-out tap62de4e08-b8 --physdev-is-bridged -j neutron-openvswi-i62de4e08-b
     76. [0:0] -A neutron-openvswi-i62de4e08-b -m state --state INVALID -j DROP
     77. [0:0] -A neutron-openvswi-i62de4e08-b -m state --state RELATED,ESTABLISHED -j RETURN
     78. [0:0] -A neutron-openvswi-i62de4e08-b -m set --match-set IPv4cf55331e-3b18-488d-8 src -j RETURN
     79. [0:0] -A neutron-openvswi-i62de4e08-b -j neutron-openvswi-sg-fallback
     80. [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-in tap62de4e08-b8 --physdev-is-bridged -j neutron-openvswi-sg-chain
     81. [0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-in tap62de4e08-b8 --physdev-is-bridged -j neutron-openvswi-o62de4e08-b
     82. [0:0] -A neutron-openvswi-INPUT -m physdev --physdev-in tap62de4e08-b8 --physdev-is-bridged -j neutron-openvswi-o62de4e08-b
     83. [0:0] -A neutron-openvswi-s62de4e08-b -m mac --mac-source fa:16:3e:bf:c7:49 -s 192.168.0.3 -j RETURN
     84. [0:0] -A neutron-openvswi-s62de4e08-b -j DROP
     85. [0:0] -A neutron-openvswi-o62de4e08-b -p udp -m udp --sport 68 --dport 67 -j RETURN
     86. [0:0] -A neutron-openvswi-o62de4e08-b -j neutron-openvswi-s62de4e08-b
     87. [0:0] -A neutron-openvswi-o62de4e08-b -p udp -m udp --sport 67 --dport 68 -j DROP
     88. [0:0] -A neutron-openvswi-o62de4e08-b -m state --state INVALID -j DROP
     89. [0:0] -A neutron-openvswi-o62de4e08-b -m state --state RELATED,ESTABLISHED -j RETURN
     90. [0:0] -A neutron-openvswi-o62de4e08-b -j RETURN
     91. [0:0] -A neutron-openvswi-o62de4e08-b -j neutron-openvswi-sg-fallback
     92. [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap7010a0ba-c0 --physdev-is-bridged -j neutron-openvswi-sg-chain
     93. [0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-out tap7010a0ba-c0 --physdev-is-bridged -j neutron-openvswi-i7010a0ba-c
     94. [0:0] -A neutron-openvswi-i7010a0ba-c -m state --state INVALID -j DROP
     95. [0:0] -A neutron-openvswi-i7010a0ba-c -m state --state RELATED,ESTABLISHED -j RETURN
     96. [0:0] -A neutron-openvswi-i7010a0ba-c -m set --match-set IPv4cbf8216f-4129-45db-b src -j RETURN
     97. [0:0] -A neutron-openvswi-i7010a0ba-c -j neutron-openvswi-sg-fallback
     98. [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-in tap7010a0ba-c0 --physdev-is-bridged -j neutron-openvswi-sg-chain
     99. [0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-in tap7010a0ba-c0 --physdev-is-bridged -j neutron-openvswi-o7010a0ba-c
    100. [0:0] -A neutron-openvswi-INPUT -m physdev --physdev-in tap7010a0ba-c0 --physdev-is-bridged -j neutron-openvswi-o7010a0ba-c
    101. [0:0] -A neutron-openvswi-s7010a0ba-c -m mac --mac-source fa:16:3e:54:9f:a8 -s 192.168.0.2 -j RETURN
    102. [0:0] -A neutron-openvswi-s7010a0ba-c -j DROP
    103. [0:0] -A neutron-openvswi-o7010a0ba-c -p udp -m udp --sport 68 --dport 67 -j RETURN
    104. [0:0] -A neutron-openvswi-o7010a0ba-c -j neutron-openvswi-s7010a0ba-c
    105. [0:0] -A neutron-openvswi-o7010a0ba-c -p udp -m udp --sport 67 --dport 68 -j DROP
    106. [0:0] -A neutron-openvswi-o7010a0ba-c -m state --state INVALID -j DROP
    107. [0:0] -A neutron-openvswi-o7010a0ba-c -m state --state RELATED,ESTABLISHED -j RETURN
    108. [0:0] -A neutron-openvswi-o7010a0ba-c -j RETURN
    109. [0:0] -A neutron-openvswi-o7010a0ba-c -j neutron-openvswi-sg-fallback
    110. [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap3d3f7a31-91 --physdev-is-bridged -j neutron-openvswi-sg-chain
    111. [0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-out tap3d3f7a31-91 --physdev-is-bridged -j neutron-openvswi-i3d3f7a31-9
    112. [0:0] -A neutron-openvswi-i3d3f7a31-9 -m state --state INVALID -j DROP
    113. [0:0] -A neutron-openvswi-i3d3f7a31-9 -m state --state RELATED,ESTABLISHED -j RETURN
    114. [0:0] -A neutron-openvswi-i3d3f7a31-9 -m set --match-set IPv4cb64a725-b0d4-4e34-8 src -j RETURN
    115. [0:0] -A neutron-openvswi-i3d3f7a31-9 -j neutron-openvswi-sg-fallback
    116. [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-in tap3d3f7a31-91 --physdev-is-bridged -j neutron-openvswi-sg-chain
    117. [0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-in tap3d3f7a31-91 --physdev-is-bridged -j neutron-openvswi-o3d3f7a31-9
    118. [0:0] -A neutron-openvswi-INPUT -m physdev --physdev-in tap3d3f7a31-91 --physdev-is-bridged -j neutron-openvswi-o3d3f7a31-9
    119. [0:0] -A neutron-openvswi-s3d3f7a31-9 -m mac --mac-source fa:16:3e:b5:e5:8c -s 192.168.0.3 -j RETURN
    120. [0:0] -A neutron-openvswi-s3d3f7a31-9 -j DROP
    121. [0:0] -A neutron-openvswi-o3d3f7a31-9 -p udp -m udp --sport 68 --dport 67 -j RETURN
    122. [0:0] -A neutron-openvswi-o3d3f7a31-9 -j neutron-openvswi-s3d3f7a31-9
    123. [0:0] -A neutron-openvswi-o3d3f7a31-9 -p udp -m udp --sport 67 --dport 68 -j DROP
    124. [0:0] -A neutron-openvswi-o3d3f7a31-9 -m state --state INVALID -j DROP
    125. [0:0] -A neutron-openvswi-o3d3f7a31-9 -m state --state RELATED,ESTABLISHED -j RETURN
    126. [0:0] -A neutron-openvswi-o3d3f7a31-9 -j RETURN
    127. [0:0] -A neutron-openvswi-o3d3f7a31-9 -j neutron-openvswi-sg-fallback
    128. [0:0] -A neutron-openvswi-sg-chain -j ACCEPT
    129. [0:0] -A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
    130. [0:0] -A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
    131. [0:0] -A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
    132. [0:0] -A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
    133. [0:0] -A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
    134. [0:0] -A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
    135. [0:0] -A FORWARD -i virbr0 -o virbr0 -j ACCEPT
    136. [0:0] -A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
    137. [0:0] -A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
    138. [0:0] -A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
    139. COMMIT
    140. # Completed on Fri Oct 10 12:57:46 2014
    141.


** Summary changed:

- neutron-openvswitch-agent fails to apply iptables rules
+ neutron-openvswitch-agent fails to apply iptables rules - Set IPv4cf55331e-3b18-488d-8 doesn't exist.

** Changed in: neutron
       Status: New => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1379779

Title:
  neutron-openvswitch-agent fails to apply iptables rules - Set
  IPv4cf55331e-3b18-488d-8 doesn't exist.

Status in OpenStack Neutron (virtual network service):
  Invalid
Status in “neutron” package in Ubuntu:
  New

Bug description:
  2014-10-10 12:49:19.947 4498 ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-4865cb3b-e783-4368-82c4-6d585ba08248 None] Error while processing VIF ports
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Traceback (most recent call last):
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1406, in rpc_loop
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     ovs_restarted)
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py", line 1205, in process_network_ports
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     port_info.get('updated', set()))
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/securitygroups_rpc.py", line 316, in setup_port_filters
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.prepare_devices_filter(new_devices)
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/securitygroups_rpc.py", line 211, in prepare_devices_filter
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     security_groups, security_group_member_ips)
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.gen.next()
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/firewall.py", line 106, in defer_apply
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.filter_defer_apply_off()
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/iptables_firewall.py", line 557, in filter_defer_apply_off
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self.iptables.defer_apply_off()
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py", line 373, in defer_apply_off
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     self._apply()
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py", line 389, in _apply
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     return self._apply_synchronized()
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py", line 444, in _apply_synchronized
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     '\n'.join(log_lines))
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/openstack/common/excutils.py", line 82, in __exit__
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     six.reraise(self.type_, self.value, self.tb)
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/iptables_manager.py", line 423, in _apply_synchronized
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     root_helper=self.root_helper)
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent   File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 84, in execute
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent     raise RuntimeError(m)
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent RuntimeError:
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'iptables-restore', '-c']
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Exit code: 2
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stdout: ''
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Stderr: "iptables-restore v1.4.21: Set IPv4cf55331e-3b18-488d-8 doesn't exist.\n\nError occurred at line: 75\nTry `iptables-restore -h' or 'iptables-restore --help' for more information.\n"
  2014-10-10 12:49:19.947 4498 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent

  ProblemType: Bug
  DistroRelease: Ubuntu 14.10
  Package: neutron-plugin-openvswitch-agent 1:2014.2~rc1-0ubuntu1
  ProcVersionSignature: User Name 3.16.0-20.27-generic 3.16.3
  Uname: Linux 3.16.0-20-generic x86_64
  ApportVersion: 2.14.7-0ubuntu5
  Architecture: amd64
  Date: Fri Oct 10 12:48:27 2014
  Ec2AMI: ami-000000af
  Ec2AMIManifest: FIXME
  Ec2AvailabilityZone: nova
  Ec2InstanceType: m1.medium
  Ec2Kernel: aki-00000002
  Ec2Ramdisk: ari-00000002
  PackageArchitecture: all
  SourcePackage: neutron
  UpgradeStatus: No upgrade log present (probably fresh install)
  modified.conffile..etc.neutron.rootwrap.d.openvswitch.plugin.filters: [deleted]

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1379779/+subscriptions
References

[Bug 1379779] [NEW] neutron-openvswitch-agent fails to apply iptables rules
From: James Page, 2014-10-10