openstack team mailing list archive

[Heiko Krämer <info@xxxxxxxxxxxxxxx>]

Heyho guys,

I've a really strange issue and i try to find out since one week why 
i've a packet loss only by one http web site.

Description:

I've setting up Neutron with L3, DHCP and Meta agent with OVS and Gre 
tunneling.
The hole network works perfectly with all required components but I've a 
strange issue.

I start an instance with ubuntu 12.04 without any problems. The instance 
can talk with WAN AND i can curl to http://rubygems.org directly on the 
command line. The gem installer cli tool (gem) is working too.

Now i'm trying to deploy *Cloudfroundry* on *OpenStack* and it works now 
for the most cases.
If i try now to deploy an rails app to cloudfoundry the warden on the 
dea will install some gems so the usual way.
The problem is i don't get data from rubygems, only from this site °°. 
The first http handshake works


curl http://rubygems.org

tcpdump in my instance but on outside of the warden container:

/09:26:33.477668 IP (tos 0x0, ttl 63, id 16829, offset 0, flags [DF], 
proto TCP (6), length 60)//
//    10.100.0.33.56832 > 54.245.255.174.80: Flags [S], cksum 0xd35c 
(correct), seq 3100954858, win 14600, options [mss 1460,sackOK,TS val 
17666391 ecr 0,nop,wscale 5], length 0//
//    0x0000:  4500 003c 41bd 4000 3f06 b8d6 0a64 0021 E..<A.@.?....d.!//
//    0x0010:  36f5 ffae de00 0050 b8d4 d0ea 0000 0000 6......P........//
//    0x0020:  a002 3908 d35c 0000 0204 05b4 0402 080a ..9..\..........//
//    0x0030:  010d 9157 0000 0000 0103 0305 ...W........//
//09:26:33.835130 IP (tos 0x0, ttl 47, id 0, offset 0, flags [DF], proto 
TCP (6), length 52)//
//    54.245.255.174.80 > 10.100.0.33.56832: Flags [S.], cksum 0x028a 
(correct), seq 325285586, ack 3100954859, win 14600, options [mss 
1460,nop,nop,sackOK,nop,wscale 7], length 0//
//    0x0000:  4500 0034 0000 4000 2f06 0a9c 36f5 ffae E..4..@./...6...//
//    0x0010:  0a64 0021 0050 de00 1363 76d2 b8d4 d0eb .d.!.P...cv.....//
//    0x0020:  8012 3908 028a 0000 0204 05b4 0101 0402 ..9.............//
//    0x0030:  0103 0307                                ....//
//09:26:33.835328 IP (tos 0x0, ttl 63, id 16830, offset 0, flags [DF], 
proto TCP (6), length 40)//
//    10.100.0.33.56832 > 54.245.255.174.80: Flags [.], cksum 0x7a9b 
(correct), seq 3100954859, ack 325285587, win 457, length 0//
//    0x0000:  4500 0028 41be 4000 3f06 b8e9 0a64 0021 E..(A.@.?....d.!//
//    0x0010:  36f5 ffae de00 0050 b8d4 d0eb 1363 76d3 6......P.....cv.//
//    0x0020:  5010 01c9 7a9b 0000 P...z...//
//09:26:33.835565 IP (tos 0x0, ttl 63, id 16831, offset 0, flags [DF], 
proto TCP (6), length 193)//
//    10.100.0.33.56832 > 54.245.255.174.80: Flags [P.], cksum 0xd305 
(correct), seq 3100954859:3100955012, ack 325285587, win 457, length 153//
//    0x0000:  4500 00c1 41bf 4000 3f06 b84f 0a64 0021 E...A.@.?..O.d.!//
//    0x0010:  36f5 ffae de00 0050 b8d4 d0eb 1363 76d3 6......P.....cv.//
//    0x0020:  5018 01c9 d305 0000 4745 5420 2f20 4854 P.......GET./.HT//
//    0x0030:  5450 2f31 2e31 0d0a 5573 6572 2d41 6765 TP/1.1..User-Age//
//    0x0040:  6e74 3a20 6375 726c 2f37 2e31 392e 3720 nt:.curl/7.19.7.//
//    0x0050:  2878 3836 5f36 342d 7063 2d6c 696e 7578 (x86_64-pc-linux//
//    0x0060:  2d67 6e75 2920 6c69 6263 7572 6c2f 372e -gnu).libcurl/7.//
//    0x0070:  3139 2e37 204f 7065 6e53 534c 2f30 2e39 19.7.OpenSSL/0.9//
//    0x0080:  2e38 6b20 7a6c 6962 2f31 2e32 2e33 2e33 .8k.zlib/1.2.3.3//
//    0x0090:  206c 6962 6964 6e2f 312e 3135 0d0a 486f .libidn/1.15..Ho//
//    0x00a0:  7374 3a20 7275 6279 6765 6d73 2e6f 7267 st:.rubygems.org//
//    0x00b0:  0d0a 4163 6365 7074 3a20 2a2f 2a0d 0a0d ..Accept:.*/*...//
//    0x00c0:  0a                                       .//
//09:26:34.025768 IP (tos 0x0, ttl 47, id 28779, offset 0, flags [DF], 
proto TCP (6), length 40)//
//    54.245.255.174.80 > 10.100.0.33.56832: Flags [.], cksum 0x7b50 
(correct), seq 325285587, ack 3100955012, win 123, length 0//
//    0x0000:  4500 0028 706b 4000 2f06 9a3c 36f5 ffae E..(pk@./..<6...//
//    0x0010:  0a64 0021 0050 de00 1363 76d3 b8d4 d184 .d.!.P...cv.....//
//    0x0020:  5010 007b 7b50 0000 P..{{P../



On the network node I get the follow responses by http correctly but 
this will not reach the vm °°.
Sometimes will reach the response the vm one min later .....


In my instance you will see some iptables rules to get connection to the 
warden containter:

iptables -L


/root@f10ee0c8-bab8-4fc1-9964-898fb76d518c:~# iptables -L//
//Chain INPUT (policy ACCEPT)//
//target     prot opt source               destination //
//
//Chain FORWARD (policy ACCEPT)//
//target     prot opt source               destination //
//warden-forward  all  --  anywhere anywhere //
//
//Chain OUTPUT (policy ACCEPT)//
//target     prot opt source               destination //
//
//Chain warden-default (1 references)//
//target     prot opt source               destination //
//
//Chain warden-forward (1 references)//
//target     prot opt source               destination //
//warden-instance-170m184dmam  all  --  anywhere anywhere            
[goto] //
//DROP       all  --  anywhere             anywhere //
//
//Chain warden-instance-170m184dmam (1 references)//
//target     prot opt source               destination //
//warden-default  all  --  anywhere anywhere            [goto]/


and *nat*:
/
//root@f10ee0c8-bab8-4fc1-9964-898fb76d518c:~# iptables -L -t nat//
//Chain PREROUTING (policy ACCEPT)//
//target     prot opt source               destination //
//warden-prerouting  all  --  anywhere anywhere //
//
//Chain INPUT (policy ACCEPT)//
//target     prot opt source               destination //
//
//Chain OUTPUT (policy ACCEPT)//
//target     prot opt source               destination //
//warden-prerouting  all  --  anywhere anywhere //
//
//Chain POSTROUTING (policy ACCEPT)//
//target     prot opt source               destination //
//warden-postrouting  all  --  anywhere anywhere //
//
//Chain warden-instance-170m184dmam (1 references)//
//target     prot opt source               destination //
//
//Chain warden-postrouting (1 references)//
//target     prot opt source               destination //
//SNAT       all  --  10.254.0.0/22        anywhere to:10.100.0.33 //
//
//Chain warden-prerouting (2 references)//
//target     prot opt source               destination //
//warden-instance-170m184dmam  all  --  anywhere anywhere //
/




I'm going to crazy because if i try to get a connection directly from 
root without the container i'll get an answer directly otherwise with 
the container the second response will reach my vm after a minute ....



I know this informations are not really helpful but I've the hope anyone 
had the same strange issue. If you need other informations please let me 
know.



Greetings
Heiko