← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1818383] [NEW] neutron not allowing access to external network

 

Public bug reported:

We did a 4 node bare metal OpenStack Queens install. After setting up
networking and adding eth0 to br-ex and restarting network service we
cannot ping from qrouter to external floating IP network. Below layout
of the 4 node setup and our ovs db info.

This was a fresh install using PackStack script modify to prep all nodes
except the storage node.

CentOS 7
OpenStack Queens release

static hostname: controller01
         Icon name: computer
        Machine ID: 0f62242dd7f04961b2fa642777708526
           Boot ID: 1bf746fe751f4e58902431573696f31e
  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-957.5.1.el7.x86_64
      Architecture: x86-64

node 1 controller/network
node 2 compute01
node 3 compute02
node 4 cinder storage


[root@controller01 neutron(keystone_admin)]# neutron-server --version
neutron-server 12.0.5

root@controller01 neutron(keystone_admin)]# ovs-vsctl show
96de914b-630f-4014-b738-e149ee385b15
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge "br-eth1"
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "br-eth1"
            Interface "br-eth1"
                type: internal
        Port "eth1"
            Interface "eth1"
        Port "phy-br-eth1"
            Interface "phy-br-eth1"
                type: patch
                options: {peer="int-br-eth1"}
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "int-br-eth1"
            Interface "int-br-eth1"
                type: patch
                options: {peer="phy-br-eth1"}
        Port patch-tun
            Interface patch-tun
                type: patch
                options: {peer=patch-int}
        Port "tap5d460e96-f2"
            tag: 1
            Interface "tap5d460e96-f2"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "qg-96178c89-7a"
            tag: 1
            Interface "qg-96178c89-7a"
                type: internal
        Port "qr-232080af-bb"
            tag: 2
            Interface "qr-232080af-bb"
                type: internal
        Port "tap31ad97cd-15"
            tag: 2
            Interface "tap31ad97cd-15"
                type: internal
    Bridge br-ex
        Port "eth0"
            Interface "eth0"
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-tun
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port patch-int
            Interface patch-int
                type: patch
                options: {peer=patch-tun}
        Port br-tun
            Interface br-tun
                type: internal
        Port "vxlan-c0a8015c"
            Interface "vxlan-c0a8015c"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.92"}
        Port "vxlan-c0a8015b"
            Interface "vxlan-c0a8015b"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.91"}
    ovs_version: "2.9.0"


floating IP network = 192.168.30.0/24

moment interface network on all nodes = 192.168.1.0/24

tenant network = 10.10.1.0/24

[root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 qg-96178c89-7a
10.10.1.0       0.0.0.0         255.255.255.0   U     0      0        0 qr-232080af-bb
192.168.1.1     0.0.0.0         255.255.255.255 UH    0      0        0 qg-96178c89-7a
192.168.30.0    0.0.0.0         255.255.255.0   U     0      0        0 qg-96178c89-7a


[root@controller01 neutron(keystone_admin)]# openstack server list
+--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+
| ID                                   | Name        | Status | Networks                                     | Image        | Flavor  |
+--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+
| e6e950c0-6efd-4b9d-913c-5736016e6a2a | Test-Cirros | ACTIVE | kubernetes-network=10.10.1.14, 192.168.30.57 | cirros image | m1.tiny |
+--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+


What's interesting, is that from the qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b we can ping the floating IP of the Test-Cirros VM, however we just cannot ping out side to any 192.168.1.0/24 IP. 


[root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b ping -c4 192.168.30.57
PING 192.168.30.57 (192.168.30.57) 56(84) bytes of data.
64 bytes from 192.168.30.57: icmp_seq=1 ttl=64 time=2.02 ms
64 bytes from 192.168.30.57: icmp_seq=2 ttl=64 time=0.500 ms
64 bytes from 192.168.30.57: icmp_seq=3 ttl=64 time=0.528 ms
64 bytes from 192.168.30.57: icmp_seq=4 ttl=64 time=0.583 ms

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1818383

Title:
  neutron not allowing access to external network

Status in neutron:
  New

Bug description:
  We did a 4 node bare metal OpenStack Queens install. After setting up
  networking and adding eth0 to br-ex and restarting network service we
  cannot ping from qrouter to external floating IP network. Below layout
  of the 4 node setup and our ovs db info.

  This was a fresh install using PackStack script modify to prep all
  nodes except the storage node.

  CentOS 7
  OpenStack Queens release

  static hostname: controller01
           Icon name: computer
          Machine ID: 0f62242dd7f04961b2fa642777708526
             Boot ID: 1bf746fe751f4e58902431573696f31e
    Operating System: CentOS Linux 7 (Core)
         CPE OS Name: cpe:/o:centos:centos:7
              Kernel: Linux 3.10.0-957.5.1.el7.x86_64
        Architecture: x86-64

  node 1 controller/network
  node 2 compute01
  node 3 compute02
  node 4 cinder storage

  
  [root@controller01 neutron(keystone_admin)]# neutron-server --version
  neutron-server 12.0.5

  root@controller01 neutron(keystone_admin)]# ovs-vsctl show
  96de914b-630f-4014-b738-e149ee385b15
      Manager "ptcp:6640:127.0.0.1"
          is_connected: true
      Bridge "br-eth1"
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "br-eth1"
              Interface "br-eth1"
                  type: internal
          Port "eth1"
              Interface "eth1"
          Port "phy-br-eth1"
              Interface "phy-br-eth1"
                  type: patch
                  options: {peer="int-br-eth1"}
      Bridge br-int
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port "int-br-eth1"
              Interface "int-br-eth1"
                  type: patch
                  options: {peer="phy-br-eth1"}
          Port patch-tun
              Interface patch-tun
                  type: patch
                  options: {peer=patch-int}
          Port "tap5d460e96-f2"
              tag: 1
              Interface "tap5d460e96-f2"
                  type: internal
          Port br-int
              Interface br-int
                  type: internal
          Port "qg-96178c89-7a"
              tag: 1
              Interface "qg-96178c89-7a"
                  type: internal
          Port "qr-232080af-bb"
              tag: 2
              Interface "qr-232080af-bb"
                  type: internal
          Port "tap31ad97cd-15"
              tag: 2
              Interface "tap31ad97cd-15"
                  type: internal
      Bridge br-ex
          Port "eth0"
              Interface "eth0"
          Port br-ex
              Interface br-ex
                  type: internal
      Bridge br-tun
          Controller "tcp:127.0.0.1:6633"
              is_connected: true
          fail_mode: secure
          Port patch-int
              Interface patch-int
                  type: patch
                  options: {peer=patch-tun}
          Port br-tun
              Interface br-tun
                  type: internal
          Port "vxlan-c0a8015c"
              Interface "vxlan-c0a8015c"
                  type: vxlan
                  options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.92"}
          Port "vxlan-c0a8015b"
              Interface "vxlan-c0a8015b"
                  type: vxlan
                  options: {df_default="true", in_key=flow, local_ip="192.168.1.90", out_key=flow, remote_ip="192.168.1.91"}
      ovs_version: "2.9.0"


  floating IP network = 192.168.30.0/24

  moment interface network on all nodes = 192.168.1.0/24

  tenant network = 10.10.1.0/24

  [root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b route -n
  Kernel IP routing table
  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 qg-96178c89-7a
  10.10.1.0       0.0.0.0         255.255.255.0   U     0      0        0 qr-232080af-bb
  192.168.1.1     0.0.0.0         255.255.255.255 UH    0      0        0 qg-96178c89-7a
  192.168.30.0    0.0.0.0         255.255.255.0   U     0      0        0 qg-96178c89-7a


  [root@controller01 neutron(keystone_admin)]# openstack server list
  +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+
  | ID                                   | Name        | Status | Networks                                     | Image        | Flavor  |
  +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+
  | e6e950c0-6efd-4b9d-913c-5736016e6a2a | Test-Cirros | ACTIVE | kubernetes-network=10.10.1.14, 192.168.30.57 | cirros image | m1.tiny |
  +--------------------------------------+-------------+--------+----------------------------------------------+--------------+---------+

  
  What's interesting, is that from the qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b we can ping the floating IP of the Test-Cirros VM, however we just cannot ping out side to any 192.168.1.0/24 IP. 

  
  [root@controller01 neutron(keystone_admin)]# ip netns exec qrouter-c2d1460b-3585-4d37-a782-0ae4a713738b ping -c4 192.168.30.57
  PING 192.168.30.57 (192.168.30.57) 56(84) bytes of data.
  64 bytes from 192.168.30.57: icmp_seq=1 ttl=64 time=2.02 ms
  64 bytes from 192.168.30.57: icmp_seq=2 ttl=64 time=0.500 ms
  64 bytes from 192.168.30.57: icmp_seq=3 ttl=64 time=0.528 ms
  64 bytes from 192.168.30.57: icmp_seq=4 ttl=64 time=0.583 ms

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1818383/+subscriptions


Follow ups