← Back to team overview

openstack team mailing list archive

Re: Instances can't reach metadata server in network HA mode

 

Are you sure your network has multi_host = True? It seems like it isn't, since the gateway listed by the guest is 10.5.5.32

In multi_host mode each node should be getting an ip from the fixed range and the guest should be using that as the gateway.

Vish

> 
> 
> 
> On Wed, Dec 19, 2012 at 1:13 PM, Vishvananda Ishaya <vishvananda@xxxxxxxxx> wrote:
> There should be a redirect in iptables from 169.254.169.254:80 to $my_ip:8775 (where nova-api-metadata is running)
> 
> So:
> 
> a) can you
> 
>   curl $my_ip:8775 (should 404)
> CloudController and Nodes awnser in the same way:
> 1.0
> 2007-01-19
> 2007-03-01
> 2007-08-29
> 2007-10-10
> 2007-12-15
> 2008-02-01
> 2008-09-01
> 2009-04-04
>  
> 
> b) if you do
> 
>   sudo iptables -t nat -L -n v
> 
> do you see the forward rule? Is it getting hit properly?
> 
> there is the correct rule, but they never got hit
> controller
>     0     0 DNAT       tcp  --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 to:200.131.6.250:8775
> 
> nodes
>     0     0 DNAT       tcp  --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 to:200.131.6.248:8775
>     0     0 DNAT       tcp  --  *      *       0.0.0.0/0            169.254.169.254      tcp dpt:80 to:200.131.6.249:8775
> 
>  
> Thanks for appearing Vish! I was wishing your help!
> 
> Vish
>  
> On Dec 19, 2012, at 6:39 AM, Gui Maluf <guimalufb@xxxxxxxxx> wrote:
> 
>> My set up is a nova-network-ha, so each of my nodes run a nova-{api-metadata,network,compute,volume}, my controller runs all of this plus the rest of things it should run.
>> Each of my nodes are the gateway for it's own instances. They all have the same net config, ip_forwarding. 
>> 
>> The main issue is that I can't telnet the nodes on port 80 that should redirect to metadatas server. metadata IP is set correctly to eth0, but 80 port is not open.
>> My doubt is, should I create a endpoint for each node api-metadata service? should I install apache on nodes?
>> 
>> I really don't know what to do anymore. This only happen on nodes, on cloudcontroller all instance run smoothly. they get the floatip, metadata service, etc.
>> 
>> Thanks in advance!
>> 
>> 
>> I will put the max of info I can here.
>> 
>> root@oxala:~# nova-manage service list                                                                                                                                                                                                                                          
>> Binary           Host                                 Zone             Status     State Updated_At
>> nova-compute     xango                                nova             enabled    :-)   2012-12-18 20:34:21
>> nova-network     xango                                nova             enabled    :-)   2012-12-18 20:34:20
>> nova-compute     oxossi                               nova             enabled    :-)   2012-12-18 20:34:15
>> nova-network     oxossi                               nova             enabled    :-)   2012-12-18 20:34:20
>> nova-volume      oxossi                               nova             enabled    :-)   2012-12-18 20:34:18
>> nova-volume      xango                                nova             enabled    :-)   2012-12-18 20:34:19
>> nova-consoleauth oxala                                nova             enabled    :-)   2012-12-18 20:34:24
>> nova-scheduler   oxala                                nova             enabled    :-)   2012-12-18 20:34:25
>> nova-cert        oxala                                nova             enabled    :-)   2012-12-18 20:34:25
>> nova-volume      oxala                                nova             enabled    :-)   2012-12-18 20:34:25
>> nova-network     oxala                                nova             enabled    :-)   2012-12-18 20:34:17
>> nova-compute     oxala                                nova             enabled    :-)   2012-12-18 20:34:10
>> 
>> controller nova.conf
>> #NETWORK
>> --allow_same_net_traffic=true
>> --network_manager=nova.network.manager.FlatDHCPManager
>> --firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
>> --public_interface=eth0
>> --flat_interface=eth1
>> --flat_network_bridge=br100
>> --fixed_range=10.5.5.32/27
>> --network_size=32
>> --flat_network_dhcp_start=10.5.5.33
>> --my_ip=200.131.6.250
>> --multi_host=True
>> #--enabled_apis=ec2,osapi_compute,osapi_volume,metadata 
>> --dhcpbridge_flagfile=/etc/nova/nova.conf
>> --dhcpbridge=/usr/bin/nova-dhcpbridge
>> --force_dhcp_release
>> --ec2_private_dns_show
>> --routing_source_ip=$my_ip
>> 
>> nodes nova.conf
>> {same network configs}
>> --my_ip=200.131.6.248
>> --multi_host=True
>> --enabled_apis=ec2,osapi_compute,osapi_volume,metadata 
>> --routing_source_ip=$my_ip
>> 
>> controller iptables -L -vn && iptables -L -vn -t nat
>> http://paste.openstack.org/show/mkWZTYI6cKHR4qUWbOUz/
>> node iptables -L -vn && iptables -L -vn -t nat
>> http://paste.openstack.org/show/28384/
>> 
>> 
>> controller ip a
>> http://paste.openstack.org/show/W2vrVtost2EP2u62iZwp/
>> root@oxala:~# route
>> Tabela de Roteamento IP do Kernel
>> Destino         Roteador        MáscaraGen.    Opções Métrica Ref   Uso Iface
>> default         200.131.6.129   0.0.0.0         UG    100    0        0 eth0
>> 10.5.5.32       *               255.255.255.224 U     0      0        0 br100
>> 200.131.6.128   *               255.255.255.128 U     0      0        0 eth0
>> 
>> node ip a
>> http://paste.openstack.org/show/S44TL3sznIztNCO3s8p2/
>> root@oxossi:~# route
>> Tabela de Roteamento IP do Kernel
>> Destino         Roteador        MáscaraGen.    Opções Métrica Ref   Uso Iface
>> default         200.131.6.129   0.0.0.0         UG    100    0        0 eth0
>> 10.5.5.32       *               255.255.255.224 U     0      0        0 br100
>> 200.131.6.128   *               255.255.255.128 U     0      0        0 eth0
>> 
>> 
>> And finnaly the error throw out by the vm when running on Nodes.
>> 
>> ci-info: lo    : 1 127.0.0.1       255.0.0.0       .
>> 
>> ci-info: eth0  : 1 10.5.5.53       255.255.255.224 fa:16:3e:69:cb:d2
>> 
>> ci-info: route-0: 0.0.0.0         10.5.5.35       0.0.0.0         eth0   UG
>> 
>> ci-info: route-1: 10.5.5.32       0.0.0.0         255.255.255.224 eth0   U
>> 
>> cloud-init start running: Tue, 18 Dec 2012 20:34:09 +0000. up 4.02 seconds
>> 
>> 2012-12-18 20:34:15,967 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [6/120s]: url error [[Errno 113] No route to host]
>> 
>> 
>> -- 
>> guilherme \n
>> \t maluf
>> _______________________________________________
>> Mailing list: https://launchpad.net/~openstack
>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
> 
> 
> 
> 
> -- 
> guilherme \n
> \t maluf


Follow ups

References