← Back to team overview

openstack team mailing list archive

Re: Quantum floating IP SNAT not working

 

Damn. Found it.
I stupidly forgot to add manual route to 192.168.1.252 for qg (gateway) network ! I had all the keys, I knew that for metadata traffic, you need external mapping to router IP, I saw that iptables was saying 'outbound traffic thru 192.168.10.1' (ie. qg - router), but I didn't noticed to add it.

Correct route is as follows :
sylvain@folsom02:~$ route -n
Table de routage IP du noyau
Destination Passerelle Genmask Indic Metric Ref Use Iface 0.0.0.0 192.168.1.252 0.0.0.0 UG 0 0 0 qg-f2397006-20
0.0.0.0         192.168.1.252   0.0.0.0         UG    0 0        0 br-ex
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 tap2ed3cd8a-03 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-f76e4668-fa
169.254.0.0     0.0.0.0         255.255.0.0     U     1000 0        0 eth0
172.16.0.0      0.0.0.0         255.255.240.0   U     0 0        0 eth0
192.168.1.252 0.0.0.0 255.255.255.255 UH 0 0 0 qg-f2397006-20
192.168.1.252   0.0.0.0         255.255.255.255 UH    0 0        0 br-ex
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-f2397006-20
192.168.10.0    0.0.0.0         255.255.255.0   U     0 0        0 br-ex

(I also removed the default entry for eth0 (mngmt interface), as it was nonsense).

-Sylvain

Le 19/02/2013 20:39, Sylvain Bauza a écrit :
Hi,

I have a classic "Provider Network, private networks" as follows :
 - internal network 10.0.0.0/24
- external network 192.168.10.0/24 gw 192.168.1.252 (I know, I have to add a manual route on both gw and network node)
 - br-ex is having 192.168.10.254

I have a floating IP 192.168.10.2 NAT'd to 10.0.0.4 and my external router IP address is 192.168.10.1

For some reason, on br-ex, I see traffic from 10.0.0.4 to 192.168.1.154 (my own computer) with tcpdump : 19:55:58.938524 00:13:8f:51:96:9d > bc:ae:c5:d6:9d:60, ethertype IPv4 (0x0800), length 98: 10.0.0.4 > 192.168.1.154: ICMP echo request, id 62465, seq 1189, length 64

I should at least see a second ICMP request from 192.168.10.2 to 192.168.1.154 (with same seq. number) if L3 agent would work correctly. DNAT is working correctly : I can ping from 192.168.1.154 to 192.168.10.2. As a consequence, I can't access Internet from my VMs, but I can reach them thru SSH.

Do you have any idea for the root cause ?
As said, the problem is bounded to a SNAT issue with Quantum L3 agent, as DNAT is working perfectly.

Please find below my iproute2 status, my route configuration, plus iptables -t nat.
sylvain@folsom02:~$ route -n
Table de routage IP du noyau
Destination Passerelle Genmask Indic Metric Ref Use Iface
0.0.0.0         192.168.1.252   0.0.0.0         UG    0 0        0 br-ex
0.0.0.0         172.16.0.2      0.0.0.0         UG    0 0        0 eth0
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 tap2ed3cd8a-03 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-f76e4668-fa 169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 eth0
172.16.0.0      0.0.0.0         255.255.240.0   U     0 0        0 eth0
192.168.1.252   0.0.0.0         255.255.255.255 UH    0 0        0 br-ex
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-f2397006-20
192.168.10.0    0.0.0.0         255.255.255.0   U     0 0        0 br-ex

sylvain@folsom02:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:13:8f:51:96:9d brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:e0:7d:b5:df:ae brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.2/20 brd 172.16.15.255 scope global eth0
    inet6 fe80::2e0:7dff:feb5:dfae/64 scope link
       valid_lft forever preferred_lft forever
6: br-ex: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 00:13:8f:51:96:9d brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.254/24 scope global br-ex
11: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether ca:0d:5d:80:15:49 brd ff:ff:ff:ff:ff:ff
12: qr-f76e4668-fa: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:4a:82:b6 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-f76e4668-fa
    inet6 fe80::f816:3eff:fe4a:82b6/64 scope link
       valid_lft forever preferred_lft forever
13: qg-f2397006-20: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:e1:95:20 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.1/24 brd 192.168.10.255 scope global qg-f2397006-20
    inet 192.168.10.2/32 brd 192.168.10.2 scope global qg-f2397006-20
    inet6 fe80::f816:3eff:fee1:9520/64 scope link
       valid_lft forever preferred_lft forever
14: tap2ed3cd8a-03: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether fa:16:3e:8f:3c:92 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 brd 10.0.0.255 scope global tap2ed3cd8a-03
    inet6 fe80::f816:3eff:fe8f:3c92/64 scope link
       valid_lft forever preferred_lft forever
15: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
    link/ether 4e:46:e7:ec:52:49 brd ff:ff:ff:ff:ff:ff

sylvain@folsom02:~$ sudo iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
quantum-l3-agent-PREROUTING  all  --  0.0.0.0/0 0.0.0.0/0

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
quantum-l3-agent-OUTPUT  all  --  0.0.0.0/0 0.0.0.0/0

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
quantum-l3-agent-POSTROUTING  all  --  0.0.0.0/0 0.0.0.0/0
quantum-postrouting-bottom  all  --  0.0.0.0/0 0.0.0.0/0

Chain quantum-l3-agent-OUTPUT (1 references)
target     prot opt source               destination
DNAT       all  --  0.0.0.0/0            192.168.10.2 to:10.0.0.4

Chain quantum-l3-agent-POSTROUTING (1 references)
target     prot opt source               destination
ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ! ctstate DNAT
ACCEPT     all  --  10.0.0.0/24          172.16.0.1

Chain quantum-l3-agent-PREROUTING (1 references)
target     prot opt source               destination
DNAT tcp -- 0.0.0.0/0 169.254.169.254 tcp dpt:80 to:172.16.0.1:8775
DNAT       all  --  0.0.0.0/0            192.168.10.2 to:10.0.0.4

Chain quantum-l3-agent-float-snat (1 references)
target     prot opt source               destination
SNAT       all  --  10.0.0.4             0.0.0.0/0 to:192.168.10.2

Chain quantum-l3-agent-snat (1 references)
target     prot opt source               destination
quantum-l3-agent-float-snat  all  --  0.0.0.0/0 0.0.0.0/0
SNAT       all  --  10.0.0.0/24          0.0.0.0/0 to:192.168.10.1

Chain quantum-postrouting-bottom (1 references)
target     prot opt source               destination
quantum-l3-agent-snat  all  --  0.0.0.0/0            0.0.0.0/0


Thanks for your help,
-Sylvain



References