openstack team mailing list archive

[Brian Haley <brian.haley@xxxxxx>]

On 03/14/2012 01:50 PM, Justin Santa Barbara wrote:
> We recently changed the MAC address assigned to guests so that they started with
> 0xfe, in the hope of avoiding (theoretical?) issues with MAC addresses changing
> on the bridge device as machines are shut down (because supposedly the bridge
> grabs the lowest MAC address numerically):
> https://bugs.launchpad.net/nova/+bug/921838

I believe the bridge changing it's MAC address is a known issue, fixed in later
versions of libvirt, see https://bugzilla.redhat.com/show_bug.cgi?id=609463

Absent of that fix, the best solution I've found is:

- Create a dummy tap device, and attach it to the bridge

	$ ip tuntap add dev tap_brfoo mode tap
	$ brctl addif brfoo tap_brfoo

- Set it's MAC to $MAC_FOO (whatever you choose)

	$ ip link set tap_brfoo address $MAC_FOO

- And the bridge's MAC too

	$ ip link set brfoo address $MAC_FOO

This should anchor the bridge's MAC address to $MAC_FOO for the life of the bridge.

You could set the bridge in promisc mode if you don't like the above, but then
you'll start seeing packets duplicated, yuck.

-Brian

> However, it looks we bumped into some similar behavior done by libvirt: It also
> sets the first byte to 0xfe for the host network device, in the hope of avoiding
> the same bug.  Thus, with the patch, the host vnetX and the guest eth0 have the
> same MAC address.  I think this breaks FlatManager, but I don't know why, and I
> really don't know why it wouldn't break other modes, and I'm hoping a network
> guru can explain/confirm.
> 
> When they have the same MAC address, ARP resolution isn't working: the guest
> issues an ARP request for the gateway, on the host I can see the ARP request and
> response, but the guest doesn't appear to see/accept the ARP response and so it
> just keeps retrying.
> 
> This message appears in dmesg:
> [ 2199.836114] br100: received packet on vnet1 with own address as source address
> 
> I'm guessing that 'address' means 'MAC address', and this is why ARP is failing,
> it sounds like the bridge might be dropping the packet.
> 
> Changing to 0x02, or 0xfc does fix it (although my arithmetic was wrong, and
> vishy points out we should use 0xfa instead of 0xfc).
> 
> Networking guru questions:
> 
>     * Does this explanation make sense?
>     * Why didn't other networking modes break?
>     * Should we simply revert the change and go back to 0x02?
>     * Should we switch to 0xfa to try to avoid the bridge interface problems?
>        Or does it simply not matter if libvirt is changing the MAC for us?
>     * Can anyone explain https://bugs.launchpad.net/nova/+bug/908194 ?  Is that
>       bug because there was no 'real' device on the bug reporter's bridge?
> 
> 
> Vishy has proposed this patch, which looks good to me:
> https://review.openstack.org/#patch,sidebyside,5351,1,nova/utils.py
> 
> 
> Justin
> 
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp