openstack team mailing list archive

[Vishvananda Ishaya <vishvananda@xxxxxxxxx>]

It is a little bit tricky to set up FlatDHCP with multiple hosts.  I've explained the issues multiple times to different people, so it seems like it is time to write it down and send it to the mailing list so that everyone has access to the information.  Maybe we can even come up with a way to make the setup easier.

The main issue is that VlatDHCP doesn't create vlans, but it does create a bridge.  This bridge works just fine on a single host, but when there are multiple hosts, traffic needs a way to get out of the bridge onto a physical interface.  Be careful with --flat_interface, if you specify an interface that already has an ip it will break.  If this is the interface you are sshing over, you will be in serious trouble unless you have ipmi/console access.

You have three options:

1. If you have an unused interface on your hosts that has connectivity with no ip address, you can simply tell FlatDHCP to bridge into the interface by specifying --flat_interface=<interface> in your flagfile.  The network host will automatically add the gateway ip to this bridge. You can also add the interface to br100 manually  and leave flat_interface alone.

2. You can create a new interface.  The easiest way to do this is to use a vlan if your switch supports it.  You should also be able to do this using a tun/tap device, but I haven't tried this method.  Once you have your new interface, you can use method 1.

3.  If you are unable/unwilling to create a new interface, you can still get it to work, but it requires a little more trickiness.  The idea is to manually create br100, put the ip of the interface onto the bridge, then attach the interface to the bridge.  If you don't have ipmi/console access and you are messing with the interface that you are using for ssh, you need all of this to happen at once.  This means (on debian at least) editing /etc/network/interfaces and moving the settings from <eth> onto br100, specifying that <eth> should be attached to br100, and doing an /etc/init.d/networking restart.  At the end of this process your bridging should be handled and nova-network will add the gateway ip to br100 (so it will have two ips)


And for improving the ease of setup:

Perhaps there is some way we can make nova smart enough to do option 3 on its own, but it seems a bit dangerous.  Another possibility I have considered is to not use a bridge at all on the nova-network host, and just make it assign the ip directly to the specified ethernet device.  Bridging on the compute hosts could still happen, because the compute hosts don't need an ip on the bridge.  I 'think' this would work, but I would have to test it to make sure.