openstack team mailing list archive

[Sharif Islam <islamsh@xxxxxxxxxxx>]

Thanks Jorge.

On 10/07/2011 02:30 PM, Jorge Luiz Correa wrote:
> It seems that configs are OK. 


Yes, that's what baffling me. I am pretty sure it was working before. I
applied some redhat update and rebooted the cluster couple weeks ago.



> 
> If you use dig from the controller, could resolv names? I'm asking
> because can be case that packets arrive from VMs to controller but
> couldn't go to Internet. 

>From the controller, it is fine:

# dig @10.0.1.1 google.com

; <<>> DiG 9.7.3-P1-RedHat-9.7.3-2.el6_1.P1.1 <<>> @10.0.1.1 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18002
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		263	IN	A	72.14.204.99
google.com.		263	IN	A	72.14.204.103
google.com.		263	IN	A	72.14.204.104
google.com.		263	IN	A	72.14.204.105
google.com.		263	IN	A	72.14.204.147

;; AUTHORITY SECTION:
google.com.		84809	IN	NS	ns2.google.com.
google.com.		84809	IN	NS	ns3.google.com.
google.com.		84809	IN	NS	ns4.google.com.
google.com.		84809	IN	NS	ns1.google.com.

;; ADDITIONAL SECTION:
ns1.google.com.		160584	IN	A	216.239.32.10
ns2.google.com.		159501	IN	A	216.239.34.10
ns3.google.com.		159500	IN	A	216.239.36.10
ns4.google.com.		159497	IN	A	216.239.38.10

;; Query time: 1 msec
;; SERVER: 10.0.1.1#53(10.0.1.1)
;; WHEN: Fri Oct  7 14:44:10 2011
;; MSG SIZE  rcvd: 244






> 
> Another thing you can check. Although the resolv.conf of VMs are set
> with 10.0.1.1, there are a lot of iptables rules. I was using Cactus and
> I noticed that. If you type nova-manage network list you will see the
> networks and you can see a DNS collumn. The default was 8.8.4.4 but when
> I started instances this values changed to 10.0.2.1 or something like
> that! My concern is about what address nova uses to create rules!! Maybe
> all services are OK but a wrong iptables rule is dropping packets!
> 
> iptables -n -L

http://paste.openstack.org/show/2646/



> iptables -n -L -t nat

http://paste.openstack.org/show/2647/

> 
> Check if you have some rule permitting udp 53 to be forward/accepted
> (ie, not dropped). 

Looks ok to me:

ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:53
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:53



> 
> As a debug option, you can run tcpdump on the controller interface and
> see what are happening with the packets.
> 
> tcpdump -n -i <interface> port 53

#  tcpdump -n -i eth0 port 53
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
15:42:41.459072 IP 10.0.1.4.46200 > 10.0.1.1.domain: 46894+ A?
google.com. (28)
15:42:41.459423 IP 10.0.1.4.49593 > 10.0.1.1.domain: 46894+ A?
google.com. (28)
15:42:41.459748 IP 10.0.1.4.32779 > 10.0.1.1.domain: 28545+ A?
google.com.novalocal. (38)
15:42:41.460029 IP 10.0.1.4.52463 > 10.0.1.1.domain: 28545+ A?
google.com.novalocal. (38)


This is when I pinged google.com from the vm. So iptables blocking
something?

--sharif