openstack team mailing list archive

[Stefano Zanella <zanella.stefano@xxxxxxxxx>]

Hi Umar,
I'm sorry again but I'm approaching a deadline on next Tuesday, so I've
little for anything.
I'd like to reproduce your environment on my test system (it's not too
distant from yours), so I can give you an accurate response.
I'll keep you updated ASAP.
Thanks for patience.
Regards,
    Stefano


On Tue, Jan 8, 2013 at 9:16 PM, Umar Draz <unix.co@xxxxxxxxx> wrote:

> HI Stefano,
>
> Thanks for your reply
>
> I can ping all nodes from their local IP using any virtual machine.
>
> 1) I have ubuntu 12.10 on all compute nodes
> 2) I don't have any iptables on all compute nodes. Nova its self intall
> iptables firewall
>
> Please find attached file as per your instructions.
>
> Best Regards,
>
> Umar
>
>
> On Wed, Jan 9, 2013 at 12:23 AM, Stefano Zanella <
> zanella.stefano@xxxxxxxxx> wrote:
>
>> Sorry for the delay, it was a busy day.
>> I'm missing a step here: are you able to ping all 3 compute nodes from a
>> VM inside one of them, or can you ping for each VM only the corresponding
>> node?
>> Can you now paste the output of:
>> ip addr list on hypervisor and VM
>> route -n on hypervisor and VM
>> brctl show on hypervisor
>> iptables -L -nv on hypervisor
>> iptables -L -nv -t nat on hypervisor
>> (I'm trying to avoid for now to track traffic with tcpdump, but it'll be
>> next step if we cannot find the problem this way)
>>
>> Do you have a standard iptables or do you have some custom rules? Also,
>> what OS are the hypervisors running on?
>> Thanks,
>>     Stefano
>>
>>
>> On Tue, Jan 8, 2013 at 12:10 PM, Umar Draz <unix.co@xxxxxxxxx> wrote:
>>
>>> Hi Stefano,
>>>
>>> No Luck, Still same,
>>>
>>> I can ping all 3 compute nodes
>>>
>>> 192.168.1.133
>>> 192.168.1.134
>>> 192.168.1.135
>>>
>>> from any virtual machine, but I can not ping, 192.168.1.136 another
>>> linux machine on local network.
>>>
>>> Best Regards,
>>>
>>> Umar
>>>
>>> On Tue, Jan 8, 2013 at 2:56 AM, Stefano Zanella <
>>> zanella.stefano@xxxxxxxxx> wrote:
>>>
>>>> I think there's a mismatching here between configuration and intended
>>>> behavior, I'm sorry not to have detected it before.
>>>> With your configuration, you're bridging (Layer 2) two different
>>>> networks (Layer3). They cannot communicate if not properly routed or
>>>> masqueraded.
>>>>
>>>> Do you need to NAT VMs directly with public IPs? If not, I'd suggest
>>>> you to change the configuration as follows:
>>>> # NETWORK
>>>> network_manager=nova.network.manager.FlatDHCPManager
>>>> force_dhcp_release=True
>>>> dhcpbridge_flagfile=/etc/nova/nova.conf
>>>> my_ip=6x.1x.84.132
>>>> public_interface=eth1
>>>> flat_network_bridge=br100
>>>> fixed_range=10.0.0.0/24
>>>>
>>>> This way, nova-network will setup NAT between 10.0.0.0/24 and
>>>> 192.168.1.0/24 and you should be able to reach your LAN. Then, if you
>>>> want to reach machines inside VMs private network, you could add a floating
>>>> IP range and assign them to VMs.
>>>> Hope this could solve the problem.
>>>> Regards,
>>>>     Stefano
>>>>
>>>>
>>>> On Mon, Jan 7, 2013 at 9:14 PM, Umar Draz <unix.co@xxxxxxxxx> wrote:
>>>>
>>>>> I did this on compute
>>>>> root@compute1:~# echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
>>>>>
>>>>> and the result from vm
>>>>> root@vm:~# ping 192.168.1.134
>>>>>
>>>>> PING 192.168.1.134 (192.168.1.134) 56(84) bytes of data.
>>>>> From 10.0.0.2 icmp_seq=1 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=2 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=3 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=4 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=5 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=6 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=7 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=8 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=9 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=10 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=11 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=12 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=13 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=14 Destination Host Unreachable
>>>>> From 10.0.0.2 icmp_seq=15 Destination Host Unreachable
>>>>> Best Regards,
>>>>>
>>>>> Umar
>>>>>
>>>>> On Tue, Jan 8, 2013 at 1:02 AM, Stefano Zanella <
>>>>> zanella.stefano@xxxxxxxxx> wrote:
>>>>>
>>>>>> Can you try to set rp_filter to 0? I needed to disable it today,
>>>>>> otherwise I was facing problem similar to yours.
>>>>>> Try to ping with rp_filter disabled, let's see if we can resolve the
>>>>>> problem that way.
>>>>>> Regards,
>>>>>>     Stefano
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 7, 2013 at 8:57 PM, Umar Draz <unix.co@xxxxxxxxx> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Here is the result
>>>>>>>
>>>>>>> root@compute1:~# cat /proc/sys/net/ipv4/ip_forward
>>>>>>> 1
>>>>>>>
>>>>>>> root@compute1:~# cat /proc/sys/net/ipv4/conf/default/rp_filter
>>>>>>> 1
>>>>>>>
>>>>>>> root@compute1:~# nova secgroup-list-rules default
>>>>>>> +-------------+-----------+---------+-----------+--------------+
>>>>>>> | IP Protocol | From Port | To Port | IP Range  | Source Group |
>>>>>>> +-------------+-----------+---------+-----------+--------------+
>>>>>>> | icmp        | -1        | -1      | 0.0.0.0/0 |              |
>>>>>>> | tcp         | 22        | 22      | 0.0.0.0/0 |              |
>>>>>>> | tcp         | 80        | 80      | 0.0.0.0/0 |              |
>>>>>>> | tcp         | 443       | 443     | 0.0.0.0/0 |              |
>>>>>>> | tcp         | 16667     | 16667   | 0.0.0.0/0 |              |
>>>>>>> +-------------+-----------+---------+-----------+--------------+
>>>>>>>
>>>>>>> Best Regards,
>>>>>>>
>>>>>>> Umar
>>>>>>> On Tue, Jan 8, 2013 at 12:52 AM, Stefano Zanella <
>>>>>>> zanella.stefano@xxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Routing and IP setup looks ok. What's the output of
>>>>>>>>   cat /proc/sys/net/ipv4/ip_forward
>>>>>>>> and
>>>>>>>>   cat /proc/sys/net/ipv4/conf/default/rp_filter
>>>>>>>>
>>>>>>>> Also, did you setup security groups correctly? What's the output of
>>>>>>>>   nova secgroup-list-rules default
>>>>>>>>
>>>>>>>> You should have setup at least a rule for allowing icmp traffic.
>>>>>>>> Thanks,
>>>>>>>>     Stefano
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jan 7, 2013 at 8:39 PM, Umar Draz <unix.co@xxxxxxxxx>wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Here is the result
>>>>>>>>>
>>>>>>>>> Compute node
>>>>>>>>> ------------
>>>>>>>>>
>>>>>>>>> *brctl show*
>>>>>>>>>
>>>>>>>>> bridge name     bridge id               STP enabled     interfaces
>>>>>>>>> br100           8000.002590976edb       no              eth1
>>>>>>>>>                                                         vnet0
>>>>>>>>> *ip addr list*
>>>>>>>>>
>>>>>>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
>>>>>>>>>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>>>>>>>>>     inet 127.0.0.1/8 scope host lo
>>>>>>>>>     inet 169.254.169.254/32 scope link lo
>>>>>>>>>     inet6 ::1/128 scope host
>>>>>>>>>        valid_lft forever preferred_lft forever
>>>>>>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
>>>>>>>>> UP qlen 1000
>>>>>>>>>     link/ether 00:25:90:97:6e:da brd ff:ff:ff:ff:ff:ff
>>>>>>>>>     inet 69.155.84.133/25 brd 85.195.84.255 scope global eth0
>>>>>>>>>     inet 69.155.84.142/32 scope global eth0
>>>>>>>>>     inet6 fe80::225:90ff:fe97:6eda/64 scope link
>>>>>>>>>        valid_lft forever preferred_lft forever
>>>>>>>>> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq
>>>>>>>>> master br100 state UP qlen 1000
>>>>>>>>>     link/ether 00:25:90:97:6e:db brd ff:ff:ff:ff:ff:ff
>>>>>>>>> 4: br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
>>>>>>>>> state UP
>>>>>>>>>     link/ether 00:25:90:97:6e:db brd ff:ff:ff:ff:ff:ff
>>>>>>>>>     inet 10.0.0.3/24 brd 10.0.0.255 scope global br100
>>>>>>>>>     inet 192.168.1.133/24 brd 192.168.1.255 scope global br100
>>>>>>>>>     inet6 fe80::225:90ff:fe97:6edb/64 scope link
>>>>>>>>>        valid_lft forever preferred_lft forever
>>>>>>>>> 9: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>>>>>>>> pfifo_fast master br100 state UNKNOWN qlen 500
>>>>>>>>>     link/ether fe:16:3e:41:0c:2a brd ff:ff:ff:ff:ff:ff
>>>>>>>>>     inet6 fe80::fc16:3eff:fe41:c2a/64 scope link
>>>>>>>>>        valid_lft forever preferred_lft forever
>>>>>>>>>
>>>>>>>>> *route -n*
>>>>>>>>>
>>>>>>>>> Kernel IP routing table
>>>>>>>>> Destination     Gateway         Genmask         Flags Metric
>>>>>>>>> Ref    Use Iface
>>>>>>>>> 0.0.0.0         69.155.84.129   0.0.0.0         UG    0
>>>>>>>>> 0        0 eth0
>>>>>>>>> 10.0.0.0        0.0.0.0         255.255.255.0   U     0
>>>>>>>>> 0        0 br100
>>>>>>>>> 69.155.84.128   0.0.0.0         255.255.255.128 U     0
>>>>>>>>> 0        0 eth1
>>>>>>>>> 192.168.1.0     0.0.0.0         255.255.255.0   U     0
>>>>>>>>> 0        0 br100
>>>>>>>>>
>>>>>>>>> *virtual machine
>>>>>>>>> ----------------------
>>>>>>>>> *
>>>>>>>>> *ip addr list*
>>>>>>>>>
>>>>>>>>> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
>>>>>>>>>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>>>>>>>>>     inet 127.0.0.1/8 scope host lo
>>>>>>>>>     inet6 ::1/128 scope host
>>>>>>>>>        valid_lft forever preferred_lft forever
>>>>>>>>> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
>>>>>>>>> pfifo_fast state UP qlen 1000
>>>>>>>>>     link/ether fa:16:3e:41:0c:2a brd ff:ff:ff:ff:ff:ff
>>>>>>>>>     inet 10.0.0.2/24 brd 10.0.0.255 scope global eth0
>>>>>>>>>     inet6 fe80::f816:3eff:fe41:c2a/64 scope link tentative
>>>>>>>>> dadfailed
>>>>>>>>>        valid_lft forever preferred_lft forever
>>>>>>>>>
>>>>>>>>> *route -n*
>>>>>>>>>
>>>>>>>>> Kernel IP routing table
>>>>>>>>> Destination     Gateway         Genmask         Flags Metric
>>>>>>>>> Ref    Use Iface
>>>>>>>>> 0.0.0.0         10.0.0.3        0.0.0.0         UG    100
>>>>>>>>> 0        0 eth0
>>>>>>>>> 10.0.0.0        0.0.0.0         255.255.255.0   U     0
>>>>>>>>> 0        0 eth0
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>>
>>>>>>>>> Umar
>>>>>>>>>
>>>>>>>>> On Tue, Jan 8, 2013 at 12:24 AM, Stefano Zanella <
>>>>>>>>> zanella.stefano@xxxxxxxxx> wrote:
>>>>>>>>>
>>>>>>>>>> Can you please post the output of "ip addr list", "route -n" and
>>>>>>>>>> "brctl show" on compute node and virtual machine? More than a firewall
>>>>>>>>>> issue, it seems a routing issue to me.
>>>>>>>>>> Thanks,
>>>>>>>>>>     Stefano
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 7, 2013 at 7:38 PM, Umar Draz <unix.co@xxxxxxxxx>wrote:
>>>>>>>>>>
>>>>>>>>>>> I think My network configuration is ok,
>>>>>>>>>>>
>>>>>>>>>>> I can ping compute's own ip address 192.168.1.133 from virtual
>>>>>>>>>>> machine. But I can't access other local machines.
>>>>>>>>>>>
>>>>>>>>>>> I think its security firewall issue or need some routing table?
>>>>>>>>>>>
>>>>>>>>>>> Here is the out put of ping.
>>>>>>>>>>>
>>>>>>>>>>> root@ubuntu-cloud# ping 192.168.1.133
>>>>>>>>>>> PING 192.168.1.133 (192.168.1.133) 56(84) bytes of data.
>>>>>>>>>>> 64 bytes from 192.168.1.133: icmp_req=1 ttl=64 time=0.225 ms
>>>>>>>>>>> 64 bytes from 192.168.1.133: icmp_req=2 ttl=64 time=0.360 ms
>>>>>>>>>>> 64 bytes from 192.168.1.133: icmp_req=3 ttl=64 time=0.271 ms
>>>>>>>>>>> root@ubuntu-cloud# ping 192.168.1.130
>>>>>>>>>>> PING 192.168.1.130 (192.168.1.130) 56(84) bytes of data.
>>>>>>>>>>> From 10.0.0.3: icmp_seq=2 Redirect Host(New nexthop:
>>>>>>>>>>> 192.168.1.130)
>>>>>>>>>>>
>>>>>>>>>>> 10.0.0.3 is the gateway of virtual machine which is the ip of
>>>>>>>>>>> compute's br100
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>>
>>>>>>>>>>> Umar
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 7, 2013 at 11:26 PM, Stefano Zanella <
>>>>>>>>>>> zanella.stefano@xxxxxxxxx> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> If you want to setup DHCP flat networking, maybe this page (and
>>>>>>>>>>>> the chapter that contains it) could help:
>>>>>>>>>>>>
>>>>>>>>>>>> http://docs.openstack.org/essex/openstack-compute/admin/content/libvirt-flat-dhcp-networking.html
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>>     Stefano
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jan 7, 2013 at 7:03 PM, Umar Draz <unix.co@xxxxxxxxx>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> my_ip=6x.1x.84.132
>>>>>>>>>>>>> public_interface=eth0
>>>>>>>>>>>>> flat_network_bridge=br100
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Umar Draz
>>>>>>>>>>> Network Architect
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Umar Draz
>>>>>>>>> Network Architect
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Umar Draz
>>>>>>> Network Architect
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Umar Draz
>>>>> Network Architect
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Umar Draz
>>> Network Architect
>>>
>>
>>
>
>
> --
> Umar Draz
> Network Architect
>