yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1377280] [NEW] Adding gateway to flat externat network breaks HA routers

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Phil Hopkins <phil.hopkins@xxxxxxxxxxxxx>
Date: Fri, 03 Oct 2014 18:45:25 -0000
Reply-to: Bug 1377280 <1377280@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

I am running Juno on Ubuntu 14.04. OpenStack is installed from source
and updated to the latest this morning. I am trying to build HA routers
using VXLAN tunnels. When I set up my external network as a VXXLAN
network type everything works properly. If I delete the VXLAN based
external network as the gateway to this network and change the external
network to a flat network by adding the flat network as the gateway
everything with the qrouter namespace disappears except for the lo
interface.

Here is my ml2_conf.ini file:
[ml2]
type_drivers = vxlan,flat
tenant_network_types = vxlan,flat
mechanism_drivers = linuxbridge,l2population
[ml2_type_flat]
flat_networks = physnet1
[ml2_type_vlan]
[ml2_type_gre]
[ml2_type_vxlan]
vni_ranges = 100:200
vxlan_group = 224.0.0.1
[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
enable_security_group = True
[agent]
l2population = True
tunnel_type = vxlan
[linuxbridge]
physical_interface_mappings = physnet1:vethOVS
[l2pop]
agent_boot_time = 180
[vxlan]
enable_vxlan = True
vxlan_group = 224.0.0.1
local_ip = 10.0.2.5
l2_population = True


contents of the qrouter namespace with external VXLAN network:
root@network:~# ip netns exec qrouter-5e9b2a5f-4431-48a0-ad31-a46c987506cf ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ha-9c3955a7-32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:10:4d:61 brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.12/18 brd 169.254.255.255 scope global ha-9c3955a7-32
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe10:4d61/64 scope link 
       valid_lft forever preferred_lft forever
3: qr-88c5895b-17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:6d:54:62 brd ff:ff:ff:ff:ff:ff
    inet 10.2.0.1/28 scope global qr-88c5895b-17
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe6d:5462/64 scope link 
       valid_lft forever preferred_lft forever
4: qr-b4a07ce5-34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:ae:b8:65 brd ff:ff:ff:ff:ff:ff
    inet 10.1.0.1/28 scope global qr-b4a07ce5-34
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feae:b865/64 scope link 
       valid_lft forever preferred_lft forever
5: qg-f454eff8-ea: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:37:3e:a9 brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.32/24 scope global qg-f454eff8-ea
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe37:3ea9/64 scope link 
       valid_lft forever preferred_lft forever

after removing the router gateway:
root@network:~# ip netns exec qrouter-a423edc7-5e12-4c15-a4eb-989c73cdb704 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ha-87269710-52: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:00:b6:9d brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.14/18 brd 169.254.255.255 scope global ha-87269710-52
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe00:b69d/64 scope link 
       valid_lft forever preferred_lft forever
3: qr-dc345670-13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:13:2b:52 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f816:3eff:fe13:2b52/64 scope link 
       valid_lft forever preferred_lft forever
4: qr-bfe17de0-2c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:04:ef:91 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f816:3eff:fe04:ef91/64 scope link 
       valid_lft forever preferred_lft forever

Everything looks good up to this point.

Building the following external network:
root@controller:~# neutron router-create MyRouter1 --distributed False --ha True
Created a new router:
+-----------------------+--------------------------------------+
| Field                 | Value                                |
+-----------------------+--------------------------------------+
| admin_state_up        | True                                 |
| distributed           | False                                |
| external_gateway_info |                                      |
| ha                    | True                                 |
| id                    | a423edc7-5e12-4c15-a4eb-989c73cdb704 |
| name                  | MyRouter1                            |
| routes                |                                      |
| status                | ACTIVE                               |
| tenant_id             | f8207c03fd1e4b4aaf123efea4662819     |
+-----------------------+--------------------------------------+
root@controller:~# neutron subnet-create --name public1-subnet public  --allocation-pool start=172.16.1.32,end=172.16.1.64 --gateway=172.16.1.6 --enable_dhcp=False 172.16.1.0/24
Created a new subnet:
+-------------------+------------------------------------------------+
| Field             | Value                                          |
+-------------------+------------------------------------------------+
| allocation_pools  | {"start": "172.16.1.32", "end": "172.16.1.64"} |
| cidr              | 172.16.1.0/24                                  |
| dns_nameservers   |                                                |
| enable_dhcp       | False                                          |
| gateway_ip        | 172.16.1.6                                     |
| host_routes       |                                                |
| id                | 2ce8e9fd-1bd8-44b6-8071-ba7ed1fd8bf4           |
| ip_version        | 4                                              |
| ipv6_address_mode |                                                |
| ipv6_ra_mode      |                                                |
| name              | public1-subnet                                 |
| network_id        | fe45f7ce-f054-4413-8106-e28a9f470f87           |
| tenant_id         | f8207c03fd1e4b4aaf123efea4662819               |
+-------------------+------------------------------------------------+

Now adding this new network as the gateway:
root@controller:~# neutron router-gateway-set MyRouter1 public1

I see this on the network node:
root@network:~# ip netns exec qrouter-a423edc7-5e12-4c15-a4eb-989c73cdb704 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1377280

Title:
  Adding gateway to flat externat network breaks HA routers

Status in OpenStack Neutron (virtual network service):
  New

Bug description:
  I am running Juno on Ubuntu 14.04. OpenStack is installed from source
  and updated to the latest this morning. I am trying to build HA
  routers using VXLAN tunnels. When I set up my external network as a
  VXXLAN network type everything works properly. If I delete the VXLAN
  based external network as the gateway to this network and change the
  external network to a flat network by adding the flat network as the
  gateway everything with the qrouter namespace disappears except for
  the lo interface.

  Here is my ml2_conf.ini file:
  [ml2]
  type_drivers = vxlan,flat
  tenant_network_types = vxlan,flat
  mechanism_drivers = linuxbridge,l2population
  [ml2_type_flat]
  flat_networks = physnet1
  [ml2_type_vlan]
  [ml2_type_gre]
  [ml2_type_vxlan]
  vni_ranges = 100:200
  vxlan_group = 224.0.0.1
  [securitygroup]
  firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
  enable_security_group = True
  [agent]
  l2population = True
  tunnel_type = vxlan
  [linuxbridge]
  physical_interface_mappings = physnet1:vethOVS
  [l2pop]
  agent_boot_time = 180
  [vxlan]
  enable_vxlan = True
  vxlan_group = 224.0.0.1
  local_ip = 10.0.2.5
  l2_population = True

  
  contents of the qrouter namespace with external VXLAN network:
  root@network:~# ip netns exec qrouter-5e9b2a5f-4431-48a0-ad31-a46c987506cf ip a
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host 
         valid_lft forever preferred_lft forever
  2: ha-9c3955a7-32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:10:4d:61 brd ff:ff:ff:ff:ff:ff
      inet 169.254.192.12/18 brd 169.254.255.255 scope global ha-9c3955a7-32
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe10:4d61/64 scope link 
         valid_lft forever preferred_lft forever
  3: qr-88c5895b-17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:6d:54:62 brd ff:ff:ff:ff:ff:ff
      inet 10.2.0.1/28 scope global qr-88c5895b-17
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe6d:5462/64 scope link 
         valid_lft forever preferred_lft forever
  4: qr-b4a07ce5-34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:ae:b8:65 brd ff:ff:ff:ff:ff:ff
      inet 10.1.0.1/28 scope global qr-b4a07ce5-34
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:feae:b865/64 scope link 
         valid_lft forever preferred_lft forever
  5: qg-f454eff8-ea: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:37:3e:a9 brd ff:ff:ff:ff:ff:ff
      inet 172.16.0.32/24 scope global qg-f454eff8-ea
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe37:3ea9/64 scope link 
         valid_lft forever preferred_lft forever

  after removing the router gateway:
  root@network:~# ip netns exec qrouter-a423edc7-5e12-4c15-a4eb-989c73cdb704 ip a
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host 
         valid_lft forever preferred_lft forever
  2: ha-87269710-52: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:00:b6:9d brd ff:ff:ff:ff:ff:ff
      inet 169.254.192.14/18 brd 169.254.255.255 scope global ha-87269710-52
         valid_lft forever preferred_lft forever
      inet6 fe80::f816:3eff:fe00:b69d/64 scope link 
         valid_lft forever preferred_lft forever
  3: qr-dc345670-13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:13:2b:52 brd ff:ff:ff:ff:ff:ff
      inet6 fe80::f816:3eff:fe13:2b52/64 scope link 
         valid_lft forever preferred_lft forever
  4: qr-bfe17de0-2c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
      link/ether fa:16:3e:04:ef:91 brd ff:ff:ff:ff:ff:ff
      inet6 fe80::f816:3eff:fe04:ef91/64 scope link 
         valid_lft forever preferred_lft forever

  Everything looks good up to this point.

  Building the following external network:
  root@controller:~# neutron router-create MyRouter1 --distributed False --ha True
  Created a new router:
  +-----------------------+--------------------------------------+
  | Field                 | Value                                |
  +-----------------------+--------------------------------------+
  | admin_state_up        | True                                 |
  | distributed           | False                                |
  | external_gateway_info |                                      |
  | ha                    | True                                 |
  | id                    | a423edc7-5e12-4c15-a4eb-989c73cdb704 |
  | name                  | MyRouter1                            |
  | routes                |                                      |
  | status                | ACTIVE                               |
  | tenant_id             | f8207c03fd1e4b4aaf123efea4662819     |
  +-----------------------+--------------------------------------+
  root@controller:~# neutron subnet-create --name public1-subnet public  --allocation-pool start=172.16.1.32,end=172.16.1.64 --gateway=172.16.1.6 --enable_dhcp=False 172.16.1.0/24
  Created a new subnet:
  +-------------------+------------------------------------------------+
  | Field             | Value                                          |
  +-------------------+------------------------------------------------+
  | allocation_pools  | {"start": "172.16.1.32", "end": "172.16.1.64"} |
  | cidr              | 172.16.1.0/24                                  |
  | dns_nameservers   |                                                |
  | enable_dhcp       | False                                          |
  | gateway_ip        | 172.16.1.6                                     |
  | host_routes       |                                                |
  | id                | 2ce8e9fd-1bd8-44b6-8071-ba7ed1fd8bf4           |
  | ip_version        | 4                                              |
  | ipv6_address_mode |                                                |
  | ipv6_ra_mode      |                                                |
  | name              | public1-subnet                                 |
  | network_id        | fe45f7ce-f054-4413-8106-e28a9f470f87           |
  | tenant_id         | f8207c03fd1e4b4aaf123efea4662819               |
  +-------------------+------------------------------------------------+

  Now adding this new network as the gateway:
  root@controller:~# neutron router-gateway-set MyRouter1 public1

  I see this on the network node:
  root@network:~# ip netns exec qrouter-a423edc7-5e12-4c15-a4eb-989c73cdb704 ip a
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host 
         valid_lft forever preferred_lft forever

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1377280/+subscriptions
Follow ups

[Bug 1377280] Re: Adding gateway to flat externat network breaks HA routers
From: Numan Siddique, 2014-10-21
[Bug 1377280] [NEW] Adding gateway to flat externat network breaks HA routers
From: Phil Hopkins, 2014-10-03
References

[Bug 1377280] [NEW] Adding gateway to flat externat network breaks HA routers
From: Phil Hopkins, 2014-10-03