openstack team mailing list archive

[John Garbutt <John.Garbutt@xxxxxxxxxx>]

I have never tried having a separate machine running nova-network as you have described. I have always had the services in VMs on XenServer/XCP.

Sorry, I assumed you had all the services running on a single machine, forgot your setup was any different.

I would suggest going towards using NetworkHA, and run nova-network inside the VM that is running nova-compute. Add the extra interface to that DomU VM, and trying using the settings I suggested.

Thanks,
John

PS
Sorry, I am bit unresponsive at the moment - on my Easter holiday before heading off to the summit

From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 04 April 2012 18:29
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

I'm able to access instance created from controller using public or private IP (Just accessing by controller itself, not external).

We have another installation with OpenStack using the same type of infrastructure and it's working (with KVM), using just one NIC on each machine. Just saying that to be clear that OpenStack itself works with ONE NIC, but XEN doesn't.

Just to remind, i have two machines:

1st Machine: Ubuntu 11.10, all nova services (except nova-compute), and glance running at 192.168.100.238.
2nd Machine: XCP 1.4.90 (IP: 192.168.100.251, Dom0) with 1 VM in PV mode running Ubuntu 11.10 with nova-compute service (IP: 192.168.100.221, DomU).

Where should I add the two VIFs? You said about DomU, but my nova-network service is running outside of Dom0/U, its correct?

I have tested adding one VIF (eth1) on DomU, but it didn't changed nothing. Also, i saw that i can ping the controller from instance or vice-versa, but i can't ping the DomU machine from the VM or virce-versa.

I have added my VIF in /etc/networking/interfaces with this sentence:
auto eth1
iface eth1 inet loopback

It's correct? I used loopback because i don't know how to add and make it active without IP.

________________________________
From: John.Garbutt@xxxxxxxxxx
To: alex_tkd@xxxxxxxx; openstack@xxxxxxxxxxxxxxxxxxx
Date: Wed, 4 Apr 2012 15:32:29 +0100
Subject: RE: [Openstack] [OpenStack] Xen Hypervisor


I would have expected to see eth0, eth1 and eth2 in your DomU after you added the extra virtual interfaces (VIFs) to the DomU VM? What change did you make?

I suspect what you have below would still conflict with what nova-network is doing with the linux bridge, but I may be wrong (anyone else got any ideas?)

Hope that helps,
John

From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 04 April 2012 14:00
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

Hi,

I have tried this but still doesn't work. Here is my interfaces config of DomU:

auto eth0:1
iface eth0:1 inet static
        address 192.168.110.1
        netmask 255.255.255.0
        broadcast 192.168.120.255
        gateway 192.168.110.1

auto eth0:2
iface eth0:2 inet static
        address 192.168.100.224
        netmask 255.255.255.0
        broadcast 192.168.100.255
        gateway 192.168.100.1


and here is ifconfig of controller, domU and dom0:

lis@controller:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 78:e7:d1:72:05:a7
          inet6 addr: fe80::7ae7:d1ff:fe72:5a7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:883334 errors:0 dropped:0 overruns:0 frame:0
          TX packets:281119 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:202144816 (202.1 MB)  TX bytes:148539506 (148.5 MB)
          Interrupt:43 Base address:0x2000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:680408 errors:0 dropped:0 overruns:0 frame:0
          TX packets:680408 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:383428479 (383.4 MB)  TX bytes:383428479 (383.4 MB)

xenbr0    Link encap:Ethernet  HWaddr 78:e7:d1:72:05:a7
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::f02c:82ff:fe1b:1abc/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:735854 errors:0 dropped:0 overruns:0 frame:0
          TX packets:280977 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:175547260 (175.5 MB)  TX bytes:148518318 (148.5 MB)

---------------------


lis@UbuntuServer:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 12:1f:9d:1f:62:4c
          inet addr:192.168.100.221  Bcast:192.168.100.255  Mask:255.255.255.0
          inet6 addr: fe80::101f:9dff:fe1f:624c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5419443 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2245580 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7458905436 (7.4 GB)  TX bytes:244815051 (244.8 MB)
          Interrupt:26

eth0:1    Link encap:Ethernet  HWaddr 12:1f:9d:1f:62:4c
          inet addr:192.168.110.1  Bcast:192.168.120.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:26

eth0:2    Link encap:Ethernet  HWaddr 12:1f:9d:1f:62:4c
          inet addr:192.168.100.224  Bcast:192.168.100.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:26

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

virbr0    Link encap:Ethernet  HWaddr 3e:86:97:11:da:21
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

---------------------

[root@localhost ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 78:E7:D1:55:CD:8C
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50546572 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25511413 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4193390563 (3.9 GiB)  TX bytes:3644720382 (3.3 GiB)
          Interrupt:254 Base address:0xc000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2110471 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2110471 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2181820646 (2.0 GiB)  TX bytes:2181820646 (2.0 GiB)

vif43.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:2246978 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5422937 errors:0 dropped:44 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:213788859 (203.8 MiB)  TX bytes:3242683616 (3.0 GiB)

vif44.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          UP BROADCAST RUNNING NOARP PROMISC  MTU:1500  Metric:1
          RX packets:849 errors:0 dropped:0 overruns:0 frame:0
          TX packets:168099 errors:0 dropped:160 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:70583 (68.9 KiB)  TX bytes:22030694 (21.0 MiB)

xenbr0    Link encap:Ethernet  HWaddr 78:E7:D1:55:CD:8C
          inet addr:192.168.100.251  Bcast:192.168.100.255  Mask:255.255.255.0
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:6053183 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5126075 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:808496916 (771.0 MiB)  TX bytes:1803944107 (1.6 GiB)


________________________________
From: John.Garbutt@xxxxxxxxxx
To: alex_tkd@xxxxxxxx; openstack@xxxxxxxxxxxxxxxxxxx
Date: Tue, 3 Apr 2012 14:13:44 +0100
Subject: RE: [Openstack] [OpenStack] Xen Hypervisor
Thanks for the info. The one thing I am missing is the ifconfig info from inside your VM instance (I would personally use XenCenter to access the console and see what is going on). I am assuming that it is not getting the correct IP address from the DHCP server in nova-network. And I am assuming there is no other DHCP server on that network.

OK, so it looks like you have this:
--flat_network_bridge=xenbr0
--flat_interface=eth0
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth0

Good news is that the DB looks to correctly have the flat_interface.

I can't see how your DomU can be listening on 192.168.100.228, given your current config0, looks like nova network has attached and reconfigured eth0 for you. Are you launching instances using horizon or the nova CLI?

One idea that might make this work (others please correct me if I am wrong):
*         Add two extra VIFs (virtual network interface) on your DomU VM
*         Attach those VIFs to xenbr0, just like the other VIF that is eth0 on your DomU

Now try the following configuration:
*         Configure eth0 to have the IP address you want (presumably it is a static address, as you can't have a DHCP on that network and use flatDHCP?): 192.168.100.238
*         Do not configure eth1 to have any address
*         Configure eth2 to have another IP address that is public facing: 10.42.0.42 / 255.255.255.0, something in the floating ip network maybe
*         Change the flags:

--flat_network_bridge=xenbr0
--flat_interface=eth1
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth2

That might leave things a little less broken than trying with a single interface in DomU?

However, I am no expert on how nova-network works, hopefully others can confirm the best way forward.

Cheers,
John

From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 03 April 2012 13:35
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

Dom0: 192.168.100.251
DomU: 192.168.100.238

nova.conf http://pastebin.com/B0PVVWiv
ifconfig (dom0 and domU): http://pastebin.com/iCLX91RS
nova network table: http://pastebin.com/k5XcXHee

Please, explain me how to take others informations if you want, i think i have been taken all information.

Thank you.

________________________________
From: John.Garbutt@xxxxxxxxxx
To: alex_tkd@xxxxxxxx; openstack@xxxxxxxxxxxxxxxxxxx
Date: Mon, 2 Apr 2012 16:05:15 +0100
Subject: RE: [Openstack] [OpenStack] Xen Hypervisor


Just double checking, but about the other machine you have put on the same network as your VMs, is the interface configured in the same subnet as the VMs?

Also, Just to be clear, not sure anyone has ever tried the sort of setup you are wanting (Single interface, with no VLANs). I have seen many setups using a single interface and VLANs, and many setups using different physical interfaces, and some combinations of the two, but I haven't seen anyone try collapsing everything onto a single physical interface. It might not work due to the way nova network in flatDHCP works (i.e. adds a DHCP server onto your network).

Just checking, but you are using FlatDHCP networking, I presume?

If you haven't assigned a floating ip and you are using some kind of flat networking, the public network configuration should be largely unimportant.

I think you want these settings:
flat_networking_bridge=xenbr0
flat_interface=eth0

What is the state of your VM, does it seem to have the correct IP address from the nova network DHCP?

I think you are close with your flags now, but I can't be specific with the help without more information:
*         a list of your flags in nova.conf
*         networking info (ifconfig or otherwise) from both Dom0 and DomU (with compute running) and the VM
*         networking config from XenServer (networks, DomU VIFs and VM VIFs)
*         a copy (in text form) of the network related tables in your DB (or all the values from your "nova-manage network create" and related calls)

More general advice is:

To make sure all the new network settings get applied, I would recommend:

 *   stop all nova services
 *   reset your nova DB

    *   delete the old DB
    *   Create the DB, do DB migration again, etc.
    *   Add in your network again, (please tell us what values you use for that)

 *   start all nova services

It might be worth adding a second VIF on the same network, calling that eth1 in the Nova domU and then using that as the flat_interface. Normally it is not recommended that you configure an IP address on the interface that nova-network uses for the guest networking. Not sure what you are trying will work.

Thanks,
John

From: openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx [mailto:openstack-bounces+john.garbutt=eu.citrix.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of Alexandre Leites
Sent: 02 April 2012 14:53
To: openstack@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Openstack] [OpenStack] Xen Hypervisor

I tried the last thing about changing public interface to eth0... but it still doesn't work. I still with erro of can't ping anything outside of VM (created by nova on xen).
> From: todd.deshane@xxxxxxx
> Date: Mon, 2 Apr 2012 09:15:52 -0400
> Subject: Re: [Openstack] [OpenStack] Xen Hypervisor
> To: alex_tkd@xxxxxxxx
> CC: openstack@xxxxxxxxxxxxxxxxxxx
>
> On Mon, Apr 2, 2012 at 8:49 AM, Alexandre Leites <alex_tkd@xxxxxxxx<mailto:alex_tkd@xxxxxxxx>> wrote:
> > Ok, anyway i tested it and didn't worked. Any other solution?
> >
>
> You should be more specific.
>
> You should explain the specific flags you tried and then post the
> relevant logs files.
>
> Thanks,
> Todd
>
>
> --
> Todd Deshane
> http://www.linkedin.com/in/deshantm
> http://blog.xen.org/
> http://wiki.xen.org/