openstack team mailing list archive

[Salvatore Orlando <Salvatore.Orlando@xxxxxxxxxxxxx>]

Hi Alexandre!

It seems John was advicing for using distinct domU interfaces (eth1 and eth2) for separating private and public traffic. This is correct, IMHO.
At a first glance, from the info you've posted, it seems that you're trying to do multi-homing on a single interface (eth0  -> eth0:1 and eth0:2).
If you're using the flags suggested by John, this is not likely to work.

Also, in case of interfaces configured with multiple IPs I seem to remember nova only takes the first IP, and however it has not mechanisms whatsoever to stating that an IP should be using for public traffic, and another IP as the gateway of the private network. However, I cannot confirm this as I don't have code at hand.

Finally, even if this is not a strict requirement, the advice is generally to use distinct Xen networks for private and public traffic. This would mean map private traffic (e.g.: eth1) to a network (e.g.: xenbr1) and then public traffic (e.g.: eth2) to another network (e.g.: xenbr2). You can also use VLANs.
However, if I'm not wrong I remember you have some constraints that prevent you from leveraging VLANs. In this case I think you can also plug both interfaces in the same network; just make sure CIDRs do not overlap and routing tables do not clash.

Regards,
Salvatore

From: openstack-bounces+salvatore.orlando=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+salvatore.orlando=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 04 April 2012 14:00
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

Hi,

I have tried this but still doesn't work. Here is my interfaces config of DomU:

auto eth0:1
iface eth0:1 inet static
        address 192.168.110.1
        netmask 255.255.255.0
        broadcast 192.168.120.255
        gateway 192.168.110.1

auto eth0:2
iface eth0:2 inet static
        address 192.168.100.224
        netmask 255.255.255.0
        broadcast 192.168.100.255
        gateway 192.168.100.1


and here is ifconfig of controller, domU and dom0:

lis@controller:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 78:e7:d1:72:05:a7
          inet6 addr: fe80::7ae7:d1ff:fe72:5a7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:883334 errors:0 dropped:0 overruns:0 frame:0
          TX packets:281119 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:202144816 (202.1 MB)  TX bytes:148539506 (148.5 MB)
          Interrupt:43 Base address:0x2000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:680408 errors:0 dropped:0 overruns:0 frame:0
          TX packets:680408 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:383428479 (383.4 MB)  TX bytes:383428479 (383.4 MB)

xenbr0    Link encap:Ethernet  HWaddr 78:e7:d1:72:05:a7
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::f02c:82ff:fe1b:1abc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:735854 errors:0 dropped:0 overruns:0 frame:0
          TX packets:280977 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:175547260 (175.5 MB)  TX bytes:148518318 (148.5 MB)

---------------------


lis@UbuntuServer:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 12:1f:9d:1f:62:4c
          inet addr:192.168.100.221  Bcast:192.168.100.255  Mask:255.255.255.0
          inet6 addr: fe80::101f:9dff:fe1f:624c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5419443 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2245580 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7458905436 (7.4 GB)  TX bytes:244815051 (244.8 MB)
          Interrupt:26

eth0:1    Link encap:Ethernet  HWaddr 12:1f:9d:1f:62:4c
          inet addr:192.168.110.1  Bcast:192.168.120.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:26

eth0:2    Link encap:Ethernet  HWaddr 12:1f:9d:1f:62:4c
          inet addr:192.168.100.224  Bcast:192.168.100.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:26

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

virbr0    Link encap:Ethernet  HWaddr 3e:86:97:11:da:21
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

---------------------

[root@localhost ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 78:E7:D1:55:CD:8C
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50546572 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25511413 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4193390563 (3.9 GiB)  TX bytes:3644720382 (3.3 GiB)
          Interrupt:254 Base address:0xc000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2110471 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2110471 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2181820646 (2.0 GiB)  TX bytes:2181820646 (2.0 GiB)

vif43.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:2246978 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5422937 errors:0 dropped:44 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:213788859 (203.8 MiB)  TX bytes:3242683616 (3.0 GiB)

vif44.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:849 errors:0 dropped:0 overruns:0 frame:0
          TX packets:168099 errors:0 dropped:160 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:70583 (68.9 KiB)  TX bytes:22030694 (21.0 MiB)

xenbr0    Link encap:Ethernet  HWaddr 78:E7:D1:55:CD:8C
          inet addr:192.168.100.251  Bcast:192.168.100.255  Mask:255.255.255.0
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:6053183 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5126075 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:808496916 (771.0 MiB)  TX bytes:1803944107 (1.6 GiB)


________________________________
From: John.Garbutt@xxxxxxxxxx
To: alex_tkd@xxxxxxxx; openstack@xxxxxxxxxxxxxxxxxxx
Date: Tue, 3 Apr 2012 14:13:44 +0100
Subject: RE: [Openstack] [OpenStack] Xen Hypervisor
Thanks for the info. The one thing I am missing is the ifconfig info from inside your VM instance (I would personally use XenCenter to access the console and see what is going on). I am assuming that it is not getting the correct IP address from the DHCP server in nova-network. And I am assuming there is no other DHCP server on that network.

OK, so it looks like you have this:
--flat_network_bridge=xenbr0
--flat_interface=eth0
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth0

Good news is that the DB looks to correctly have the flat_interface.

I can't see how your DomU can be listening on 192.168.100.228, given your current config0, looks like nova network has attached and reconfigured eth0 for you. Are you launching instances using horizon or the nova CLI?

One idea that might make this work (others please correct me if I am wrong):
*         Add two extra VIFs (virtual network interface) on your DomU VM
*         Attach those VIFs to xenbr0, just like the other VIF that is eth0 on your DomU

Now try the following configuration:
*         Configure eth0 to have the IP address you want (presumably it is a static address, as you can't have a DHCP on that network and use flatDHCP?): 192.168.100.238
*         Do not configure eth1 to have any address
*         Configure eth2 to have another IP address that is public facing: 10.42.0.42 / 255.255.255.0, something in the floating ip network maybe
*         Change the flags:

--flat_network_bridge=xenbr0
--flat_interface=eth1
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth2

That might leave things a little less broken than trying with a single interface in DomU?

However, I am no expert on how nova-network works, hopefully others can confirm the best way forward.

Cheers,
John

From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 03 April 2012 13:35
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

Dom0: 192.168.100.251
DomU: 192.168.100.238

nova.conf http://pastebin.com/B0PVVWiv
ifconfig (dom0 and domU): http://pastebin.com/iCLX91RS
nova network table: http://pastebin.com/k5XcXHee

Please, explain me how to take others informations if you want, i think i have been taken all information.

Thank you.

________________________________
From: John.Garbutt@xxxxxxxxxx
To: alex_tkd@xxxxxxxx; openstack@xxxxxxxxxxxxxxxxxxx
Date: Mon, 2 Apr 2012 16:05:15 +0100
Subject: RE: [Openstack] [OpenStack] Xen Hypervisor

Just double checking, but about the other machine you have put on the same network as your VMs, is the interface configured in the same subnet as the VMs?

Also, Just to be clear, not sure anyone has ever tried the sort of setup you are wanting (Single interface, with no VLANs). I have seen many setups using a single interface and VLANs, and many setups using different physical interfaces, and some combinations of the two, but I haven't seen anyone try collapsing everything onto a single physical interface. It might not work due to the way nova network in flatDHCP works (i.e. adds a DHCP server onto your network).

Just checking, but you are using FlatDHCP networking, I presume?

If you haven't assigned a floating ip and you are using some kind of flat networking, the public network configuration should be largely unimportant.

I think you want these settings:
flat_networking_bridge=xenbr0
flat_interface=eth0

What is the state of your VM, does it seem to have the correct IP address from the nova network DHCP?

I think you are close with your flags now, but I can't be specific with the help without more information:
*         a list of your flags in nova.conf
*         networking info (ifconfig or otherwise) from both Dom0 and DomU (with compute running) and the VM
*         networking config from XenServer (networks, DomU VIFs and VM VIFs)
*         a copy (in text form) of the network related tables in your DB (or all the values from your "nova-manage network create" and related calls)

More general advice is:

To make sure all the new network settings get applied, I would recommend:

 *   stop all nova services
 *   reset your nova DB

    *   delete the old DB
    *   Create the DB, do DB migration again, etc.
    *   Add in your network again, (please tell us what values you use for that)

 *   start all nova services

It might be worth adding a second VIF on the same network, calling that eth1 in the Nova domU and then using that as the flat_interface. Normally it is not recommended that you configure an IP address on the interface that nova-network uses for the guest networking. Not sure what you are trying will work.

Thanks,
John

From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 02 April 2012 14:53
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

I tried the last thing about changing public interface to eth0... but it still doesn't work. I still with erro of can't ping anything outside of VM (created by nova on xen).
> From: todd.deshane@xxxxxxx
> Date: Mon, 2 Apr 2012 09:15:52 -0400
> Subject: Re: [Openstack] [OpenStack] Xen Hypervisor
> To: alex_tkd@xxxxxxxx
> CC: openstack@xxxxxxxxxxxxxxxxxxx
>
> On Mon, Apr 2, 2012 at 8:49 AM, Alexandre Leites <alex_tkd@xxxxxxxx<mailto:alex_tkd@xxxxxxxx>> wrote:
> > Ok, anyway i tested it and didn't worked. Any other solution?
> >
>
> You should be more specific.
>
> You should explain the specific flags you tried and then post the
> relevant logs files.
>
> Thanks,
> Todd
>
>
> --
> Todd Deshane
> http://www.linkedin.com/in/deshantm
> http://blog.xen.org/
> http://wiki.xen.org/