yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #37907
[Bug 1491961] [NEW] On systems with no ports in br-ex, ha network is dropped when external gateway is set for ha router
Public bug reported:
when a system is brought up using vxlan which has no interfaces added
to br-ex bridge, if an ha router is created, the tunnel between the
network nodes is created, however, when a external gateways is set for
the router the vxlan tunnels between the network nodes is dropped.
how to reproduce:
bring up an openstack system using three nodes: one "controller" running q-vpn, q-agt, q-svc, and the other necessary services (keystone, mysql, etc...) , One network node running q-l3, and q-agt, and ONE compute node, running the usual nova stuff (n-cpu, etc..) and the q-l3 and q-agt
The q-l3 agent on the network node and controllers should be se to
agent_mode=dvr_snat, the l3 agent on the compute node should be set to
agent_mode=dvr
Delete all resources (routers, networks, subnets, etc...) so the system is cmpletey clean
Add the following commands to create the necessary networks.
make sure that there is not interfaces in the br-ex bridges of any of the nodes:
(ovs-vsctl show should sho br-ex bridge with no interfaces)
neutron net-create public --router:exernal
neutron subnet-create public 123.0.0.0/24 --disable-dhcp
#create the ha router:
neutron router-create harouter --ha=True --distributed=False
#check that the ha network has been created: neutron net-list
#check that the tunnels between the controller and network nodes are up: execute "sudo ovs-vsctl show " on the nodes.
# since the controller and network node have an instance of the ha router running they will connect to each other to provide
# the ha network via tunnels.
# it will look something like:
# Bridge br-tun
# fail_mode: secure
# Port patch-int
# Interface patch-int
# type: patch
# options: {peer=patch-tun}
# Port br-tun
# Interface br-tun
# type: internal
# Port "vxlan-650000b1"
# Interface "vxlan-650000b1"
# type: vxlan
# options: {df_default="true", in_key=flow, local_ip="101.0.0.178", out_key=flow, remote_ip="101.0.0.177"}
# Bridge br-ex
# ...
# next set the external gateway for the router:
neutron router-gateway-set harouter public
# at this point the vxlan tunnels above will drop (disappear from the ovs-vsctl show output)
#
# remove the external gateay from the router:
neutron router-gateway-clear harouter
# it might be necessary to execute the "neutron router-gateway-set routerha public / neutron router-gateway-clear harouter"
a few times to see the problem.
More testing suggests that adding an interface to br-ex (by hand) prevents the problem from happening.
in other words if before you do anything you do something like "sudo
ovs-vsctl add-pr br-ex dummy-interface" to each of the nodes, the
problem will not manifest.
This would suggest that the problem is only present when br-ex has no interfaces in it to begin with.
That case is very unlikey since most of the time br-ex will have an interface attached to it (the one connecting to the outside world external interface/nice).
However it could present problems for ci testing if no interface is added to br-ex.
This was last tested no master branch commit:
commit 43710925db0523dfbf0cdabbf2352db4304c6163
Merge: 397fc4d 2759362
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date: Wed Sep 2 11:20:18 2015 +0000
Merge "Remove duplicated codes in two test cases"
attaching local.conf files which can be used to setup system. change ip
addresses respectively
** Affects: neutron
Importance: Undecided
Status: New
** Tags: ha neutron routing
** Attachment added: "compute node local.conf"
https://bugs.launchpad.net/bugs/1491961/+attachment/4456949/+files/compute_node_only.conf
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1491961
Title:
On systems with no ports in br-ex, ha network is dropped when external
gateway is set for ha router
Status in neutron:
New
Bug description:
when a system is brought up using vxlan which has no interfaces added
to br-ex bridge, if an ha router is created, the tunnel between the
network nodes is created, however, when a external gateways is set for
the router the vxlan tunnels between the network nodes is dropped.
how to reproduce:
bring up an openstack system using three nodes: one "controller" running q-vpn, q-agt, q-svc, and the other necessary services (keystone, mysql, etc...) , One network node running q-l3, and q-agt, and ONE compute node, running the usual nova stuff (n-cpu, etc..) and the q-l3 and q-agt
The q-l3 agent on the network node and controllers should be se to
agent_mode=dvr_snat, the l3 agent on the compute node should be set to
agent_mode=dvr
Delete all resources (routers, networks, subnets, etc...) so the system is cmpletey clean
Add the following commands to create the necessary networks.
make sure that there is not interfaces in the br-ex bridges of any of the nodes:
(ovs-vsctl show should sho br-ex bridge with no interfaces)
neutron net-create public --router:exernal
neutron subnet-create public 123.0.0.0/24 --disable-dhcp
#create the ha router:
neutron router-create harouter --ha=True --distributed=False
#check that the ha network has been created: neutron net-list
#check that the tunnels between the controller and network nodes are up: execute "sudo ovs-vsctl show " on the nodes.
# since the controller and network node have an instance of the ha router running they will connect to each other to provide
# the ha network via tunnels.
# it will look something like:
# Bridge br-tun
# fail_mode: secure
# Port patch-int
# Interface patch-int
# type: patch
# options: {peer=patch-tun}
# Port br-tun
# Interface br-tun
# type: internal
# Port "vxlan-650000b1"
# Interface "vxlan-650000b1"
# type: vxlan
# options: {df_default="true", in_key=flow, local_ip="101.0.0.178", out_key=flow, remote_ip="101.0.0.177"}
# Bridge br-ex
# ...
# next set the external gateway for the router:
neutron router-gateway-set harouter public
# at this point the vxlan tunnels above will drop (disappear from the ovs-vsctl show output)
#
# remove the external gateay from the router:
neutron router-gateway-clear harouter
# it might be necessary to execute the "neutron router-gateway-set routerha public / neutron router-gateway-clear harouter"
a few times to see the problem.
More testing suggests that adding an interface to br-ex (by hand) prevents the problem from happening.
in other words if before you do anything you do something like "sudo
ovs-vsctl add-pr br-ex dummy-interface" to each of the nodes, the
problem will not manifest.
This would suggest that the problem is only present when br-ex has no interfaces in it to begin with.
That case is very unlikey since most of the time br-ex will have an interface attached to it (the one connecting to the outside world external interface/nice).
However it could present problems for ci testing if no interface is added to br-ex.
This was last tested no master branch commit:
commit 43710925db0523dfbf0cdabbf2352db4304c6163
Merge: 397fc4d 2759362
Author: Jenkins <jenkins@xxxxxxxxxxxxxxxxxxxx>
Date: Wed Sep 2 11:20:18 2015 +0000
Merge "Remove duplicated codes in two test cases"
attaching local.conf files which can be used to setup system. change
ip addresses respectively
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1491961/+subscriptions
