yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #38314
[Bug 1494039] [NEW] Must audit SG chains on ovs-agent restart
Public bug reported:
I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall.
I run into situations, where, the iptables SG chains/rules are
inconsistent with ovs-ports, and system interfaces - see below for an
example. In these situations, when I restart neutron-ovs-agent, I expect
that such inconsistencies are cleaned up. Arguably, agent should not
allow such situations to happen (but thats expecting code to be perfect)
By design, the neutron-ovs-agent must audit hw-resources (iptables in
this case) during startup , and cleanup inconsistencies/deltas between
whats in iptables vs whats in the control plane (ports)
The audit can look something like this:
1. IptablesManager recovers iptables chains during startup and marks all of them as "To Be Audited".
2. NeutronOvsAgent programs firewall rules for devices during startup
3. As chains are programmed, Iptables clears the "to be audited" state
4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed.
----- Inconsistency between OVS ports, System Interfaces and Iptables
[root@rhel7-25 agent]# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage
-N neutron-openvswi-ie3cb2e38-a
-N neutron-openvswi-local
-N neutron-openvswi-o76845da1-5
-N neutron-openvswi-oe3cb2e38-a
-N neutron-openvswi-s76845da1-5
-N neutron-openvswi-se3cb2e38-a
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
[root@rhel7-25 agent]# ovs-vsctl show
ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Bridge br-tun
fail_mode: secure
Port "vxlan-0a0a0a1e"
Interface "vxlan-0a0a0a1e"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"}
Port "vxlan-0a0a0a17"
Interface "vxlan-0a0a0a17"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "vxlan-0a0a0a15"
Interface "vxlan-0a0a0a15"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.3.0"
[root@rhel7-25 agent]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff
8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff
29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff
29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff
20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff
** Affects: neutron
Importance: Undecided
Status: New
** Description changed:
-
I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall.
- I run into situations, where, the iptables SG chains/rules are inconsistent with ovs-ports see below for an example.
- In these situations, when I restart neutron-ovs-agent, I expect that such inconsistencies are cleaned up.
- Arguably, such situations should never arise (but thats expecting code to be perfect)
+ I run into situations, where, the iptables SG chains/rules are
+ inconsistent with ovs-ports, and system interfaces - see below for an
+ example. In these situations, when I restart neutron-ovs-agent, I expect
+ that such inconsistencies are cleaned up. Arguably, such situations
+ should never arise (but thats expecting code to be perfect)
By design, the neutron-ovs-agent must audit hw-resources (iptables in
this case) during startup , and cleanup inconsistencies/deltas.
The audit can look something like this:
- 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited".
- 2. NeutronOvsAgent programs firewall rules for devices during startup
- 3. As chains are programmed, Iptables clears the "to be audited" state
- 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed.
+ 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited".
+ 2. NeutronOvsAgent programs firewall rules for devices during startup
+ 3. As chains are programmed, Iptables clears the "to be audited" state
+ 4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed.
----- Inconsistency between OVS ports, System Interfaces and Iptables
[root@rhel7-25 agent]# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage
-N neutron-openvswi-ie3cb2e38-a
-N neutron-openvswi-local
-N neutron-openvswi-o76845da1-5
-N neutron-openvswi-oe3cb2e38-a
-N neutron-openvswi-s76845da1-5
-N neutron-openvswi-se3cb2e38-a
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
-
[root@rhel7-25 agent]# ovs-vsctl show
ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697
- Bridge br-int
- fail_mode: secure
- Port patch-tun
- Interface patch-tun
- type: patch
- options: {peer=patch-int}
- Port br-int
- Interface br-int
- type: internal
- Bridge br-tun
- fail_mode: secure
- Port "vxlan-0a0a0a1e"
- Interface "vxlan-0a0a0a1e"
- type: vxlan
- options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"}
- Port "vxlan-0a0a0a17"
- Interface "vxlan-0a0a0a17"
- type: vxlan
- options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"}
- Port patch-int
- Interface patch-int
- type: patch
- options: {peer=patch-tun}
- Port "vxlan-0a0a0a15"
- Interface "vxlan-0a0a0a15"
- type: vxlan
- options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"}
- Port br-tun
- Interface br-tun
- type: internal
- ovs_version: "2.3.0"
+ Bridge br-int
+ fail_mode: secure
+ Port patch-tun
+ Interface patch-tun
+ type: patch
+ options: {peer=patch-int}
+ Port br-int
+ Interface br-int
+ type: internal
+ Bridge br-tun
+ fail_mode: secure
+ Port "vxlan-0a0a0a1e"
+ Interface "vxlan-0a0a0a1e"
+ type: vxlan
+ options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"}
+ Port "vxlan-0a0a0a17"
+ Interface "vxlan-0a0a0a17"
+ type: vxlan
+ options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"}
+ Port patch-int
+ Interface patch-int
+ type: patch
+ options: {peer=patch-tun}
+ Port "vxlan-0a0a0a15"
+ Interface "vxlan-0a0a0a15"
+ type: vxlan
+ options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"}
+ Port br-tun
+ Interface br-tun
+ type: internal
+ ovs_version: "2.3.0"
[root@rhel7-25 agent]# ip link
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
- link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+ 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
+ link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000
- link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff
+ link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
- link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff
+ link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
- link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff
+ link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
- link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff
+ link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
- link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff
+ link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
- link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff
+ link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff
8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
- link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff
- 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
- link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff
- 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
- link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff
- 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
- link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff
+ link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff
+ 29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
+ link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff
+ 29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
+ link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff
+ 20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
+ link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff
** Description changed:
I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall.
I run into situations, where, the iptables SG chains/rules are
inconsistent with ovs-ports, and system interfaces - see below for an
example. In these situations, when I restart neutron-ovs-agent, I expect
that such inconsistencies are cleaned up. Arguably, such situations
should never arise (but thats expecting code to be perfect)
By design, the neutron-ovs-agent must audit hw-resources (iptables in
- this case) during startup , and cleanup inconsistencies/deltas.
+ this case) during startup , and cleanup inconsistencies/deltas between
+ whats in iptables vs whats in the control plane (ports)
The audit can look something like this:
1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited".
2. NeutronOvsAgent programs firewall rules for devices during startup
3. As chains are programmed, Iptables clears the "to be audited" state
4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed.
----- Inconsistency between OVS ports, System Interfaces and Iptables
[root@rhel7-25 agent]# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage
-N neutron-openvswi-ie3cb2e38-a
-N neutron-openvswi-local
-N neutron-openvswi-o76845da1-5
-N neutron-openvswi-oe3cb2e38-a
-N neutron-openvswi-s76845da1-5
-N neutron-openvswi-se3cb2e38-a
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
[root@rhel7-25 agent]# ovs-vsctl show
ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Bridge br-tun
fail_mode: secure
Port "vxlan-0a0a0a1e"
Interface "vxlan-0a0a0a1e"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"}
Port "vxlan-0a0a0a17"
Interface "vxlan-0a0a0a17"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "vxlan-0a0a0a15"
Interface "vxlan-0a0a0a15"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.3.0"
[root@rhel7-25 agent]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff
8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff
29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff
29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff
20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff
** Description changed:
I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall.
I run into situations, where, the iptables SG chains/rules are
inconsistent with ovs-ports, and system interfaces - see below for an
example. In these situations, when I restart neutron-ovs-agent, I expect
that such inconsistencies are cleaned up. Arguably, such situations
should never arise (but thats expecting code to be perfect)
By design, the neutron-ovs-agent must audit hw-resources (iptables in
this case) during startup , and cleanup inconsistencies/deltas between
whats in iptables vs whats in the control plane (ports)
The audit can look something like this:
- 1. IptablesManager recovers iptables chains during startup and marks all them as "To Be Audited".
+ 1. IptablesManager recovers iptables chains during startup and marks all of them as "To Be Audited".
2. NeutronOvsAgent programs firewall rules for devices during startup
3. As chains are programmed, Iptables clears the "to be audited" state
4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed.
----- Inconsistency between OVS ports, System Interfaces and Iptables
[root@rhel7-25 agent]# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage
-N neutron-openvswi-ie3cb2e38-a
-N neutron-openvswi-local
-N neutron-openvswi-o76845da1-5
-N neutron-openvswi-oe3cb2e38-a
-N neutron-openvswi-s76845da1-5
-N neutron-openvswi-se3cb2e38-a
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
[root@rhel7-25 agent]# ovs-vsctl show
ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Bridge br-tun
fail_mode: secure
Port "vxlan-0a0a0a1e"
Interface "vxlan-0a0a0a1e"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"}
Port "vxlan-0a0a0a17"
Interface "vxlan-0a0a0a17"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "vxlan-0a0a0a15"
Interface "vxlan-0a0a0a15"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.3.0"
[root@rhel7-25 agent]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff
8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff
29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff
29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff
20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1494039
Title:
Must audit SG chains on ovs-agent restart
Status in neutron:
New
Bug description:
I am running Kilo 2015.1.0, with neutron-OVS, and iptables firewall.
I run into situations, where, the iptables SG chains/rules are
inconsistent with ovs-ports, and system interfaces - see below for an
example. In these situations, when I restart neutron-ovs-agent, I
expect that such inconsistencies are cleaned up. Arguably, agent
should not allow such situations to happen (but thats expecting code
to be perfect)
By design, the neutron-ovs-agent must audit hw-resources (iptables in
this case) during startup , and cleanup inconsistencies/deltas between
whats in iptables vs whats in the control plane (ports)
The audit can look something like this:
1. IptablesManager recovers iptables chains during startup and marks all of them as "To Be Audited".
2. NeutronOvsAgent programs firewall rules for devices during startup
3. As chains are programmed, Iptables clears the "to be audited" state
4. Chains which still have the "To Be Audited" flag at the end of the init phase are removed.
----- Inconsistency between OVS ports, System Interfaces and Iptables
[root@rhel7-25 agent]# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N neutron-filter-top
-N neutron-openvswi-FORWARD
-N neutron-openvswi-INPUT
-N neutron-openvswi-OUTPUT
-N neutron-openvswi-i76845da1-5 <<<<<<<< inconsistent chains, these remain as garbage
-N neutron-openvswi-ie3cb2e38-a
-N neutron-openvswi-local
-N neutron-openvswi-o76845da1-5
-N neutron-openvswi-oe3cb2e38-a
-N neutron-openvswi-s76845da1-5
-N neutron-openvswi-se3cb2e38-a
-N neutron-openvswi-sg-chain
-N neutron-openvswi-sg-fallback
[root@rhel7-25 agent]# ovs-vsctl show
ce7f5dac-9d4d-4354-9cfd-4d94dfaf1697
Bridge br-int
fail_mode: secure
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Bridge br-tun
fail_mode: secure
Port "vxlan-0a0a0a1e"
Interface "vxlan-0a0a0a1e"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.30"}
Port "vxlan-0a0a0a17"
Interface "vxlan-0a0a0a17"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.23"}
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port "vxlan-0a0a0a15"
Interface "vxlan-0a0a0a15"
type: vxlan
options: {csum="true", df_default="true", in_key=flow, local_ip="10.10.10.25", out_key=flow, remote_ip="10.10.10.21"}
Port br-tun
Interface br-tun
type: internal
ovs_version: "2.3.0"
[root@rhel7-25 agent]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:20 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether a0:36:9f:09:2c:21 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:60 brd ff:ff:ff:ff:ff:ff
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:64 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:62 brd ff:ff:ff:ff:ff:ff
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 34:40:b5:e5:68:66 brd ff:ff:ff:ff:ff:ff
8: usb0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN mode DEFAULT qlen 1000
link/ether 36:40:b5:e8:b4:37 brd ff:ff:ff:ff:ff:ff
29133: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 56:bf:95:17:ad:4d brd ff:ff:ff:ff:ff:ff
29134: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 66:6a:80:04:d8:4b brd ff:ff:ff:ff:ff:ff
20695: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
link/ether 6e:c5:a4:c5:89:b5 brd ff:ff:ff:ff:ff:ff
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1494039/+subscriptions
Follow ups
