yahoo-eng-team team mailing list archive

[scsnow <code-red@xxxxxxxxx>]

Public bug reported:

I'm running Havana multinode (Neutron LB plugin). By default Openstack
sets hairpin_mode for VM ports connected to bridge:

# cat /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode
1

With this flag set broadcast packets are reflected back from bridge to
interface, where they came from. That's do not cause any problems until
you have bridges inside VM.

Network diagram:
VM1  [SIMICS] -> [tap01] -> [br0] -> [eth0] -> 
HOST -> [tap6cfe8817-2a] -> [brqd1485e0f-cc] -> [tapXXX] -> 
VM2  -> [eth0] -> [dhcpd]

Simics (http://en.wikipedia.org/wiki/Simics) simulates Linux box with
internal network interface attached to tap01. Usually we have more than
one taps. DHCP client inside Simics requests IP address from DHCP server
running on VM2.

DHCP request packet with MAC address XYZ of network adapter that exists
within Simics instance is reflected back from [brqd1485e0f-cc] to
[tap6cfe8817-2a]. When it reaches br0 again, br0 rewrites own MAC
learning table: packet is came from eth0 side, so device with MAC
address XYZ is attached to the same port as eth0.

Anyway original DHCP request reaches dhcpd running on VM2. DHCP reply is
sent back to MAC XYZ and it comes to br0 on VM1. br0 remembered that
device with MAC XYZ is attached to the same port as eth0, so this packet
is sent back to eth0, but not to tap01, where it was originated from in
fact. So network adapter inside Simics will never get DHCP reply from
dhcpd...

Workaround for this issue:
1) $ brctl setageing br0 0 # disables MAC learning on br0
2) Manually disable hairpin_mode for VM ports: echo 0 > /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode

However, it's not acceptable for our environment to disable MAC learning
on br0, so we commented out strings in openstack sources, that enables
hairpin_mode.

As far as I understand hairpin_mode is only needed to allow VM ping
itself with floating ip address. However, we have no floating address
assigned to that VM. So I suggest to control hairpin_mode for VM ports
in nova.conf:

hairpin_mode = 1 # by default

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: hairpin

** Description changed:

  I'm running Havana multinode (Neutron LB plugin). By default Openstack
  sets hairpin_mode for VM ports connected to bridge:
  
- # cat /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode 
+ # cat /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode
  1
  
  With this flag set broadcast packets are reflected back from bridge to
  interface, where they came from. That's do not cause any problems until
  you have bridges inside VM.
  
  Network diagram:
- |========== VM1 ===============|==================== HOST =================|===== VM2 ======|
- [SIMICS] -> [tap01] -> [br0] -> [eth0] -> [tap6cfe8817-2a] -> [brqd1485e0f-cc] -> [tapXXX] -> [eth0] -> [dhcpd]
+ VM1  [SIMICS] -> [tap01] -> [br0] -> [eth0] -> 
+ HOST -> [tap6cfe8817-2a] -> [brqd1485e0f-cc] -> [tapXXX] -> 
+ VM2  -> [eth0] -> [dhcpd]
  
  Simics (http://en.wikipedia.org/wiki/Simics) simulates Linux box with
  internal network interface attached to tap01. Usually we have more than
  one taps. DHCP client inside Simics requests IP address from DHCP server
  running on VM2.
  
  DHCP request packet with MAC address XYZ of network adapter that exists
  within Simics instance is reflected back from [brqd1485e0f-cc] to
  [tap6cfe8817-2a]. When it reaches br0 again, br0 rewrites own MAC
  learning table: packet is came from eth0 side, so device with MAC
  address XYZ is attached to the same port as eth0.
  
  Anyway original DHCP request reaches dhcpd running on VM2. DHCP reply is
  sent back to MAC XYZ and it comes to br0 on VM1. br0 remembered that
  device with MAC XYZ is attached to the same port as eth0, so this packet
  is sent back to eth0, but not to tap01, where it was originated from in
  fact. So network adapter inside Simics will never get DHCP reply from
  dhcpd...
  
  Workaround for this issue:
  1) $ brctl setageing br0 0 # disables MAC learning on br0
  2) Manually disable hairpin_mode for VM ports: echo 0 > /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode
  
  However, it's not acceptable for our environment to disable MAC learning
  on br0, so we commented out strings in openstack sources, that enables
  hairpin_mode.
  
  As far as I understand hairpin_mode is only needed to allow VM ping
  itself with floating ip address. However, we have no floating address
  assigned to that VM. So I suggest to control hairpin_mode for VM ports
  in nova.conf:
  
  hairpin_mode = 1 # by default

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1264308

Title:
  hairpin_mode should be configurable in config file

Status in OpenStack Compute (Nova):
  New

Bug description:
  I'm running Havana multinode (Neutron LB plugin). By default Openstack
  sets hairpin_mode for VM ports connected to bridge:

  # cat /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode
  1

  With this flag set broadcast packets are reflected back from bridge to
  interface, where they came from. That's do not cause any problems
  until you have bridges inside VM.

  Network diagram:
  VM1  [SIMICS] -> [tap01] -> [br0] -> [eth0] -> 
  HOST -> [tap6cfe8817-2a] -> [brqd1485e0f-cc] -> [tapXXX] -> 
  VM2  -> [eth0] -> [dhcpd]

  Simics (http://en.wikipedia.org/wiki/Simics) simulates Linux box with
  internal network interface attached to tap01. Usually we have more
  than one taps. DHCP client inside Simics requests IP address from DHCP
  server running on VM2.

  DHCP request packet with MAC address XYZ of network adapter that
  exists within Simics instance is reflected back from [brqd1485e0f-cc]
  to [tap6cfe8817-2a]. When it reaches br0 again, br0 rewrites own MAC
  learning table: packet is came from eth0 side, so device with MAC
  address XYZ is attached to the same port as eth0.

  Anyway original DHCP request reaches dhcpd running on VM2. DHCP reply
  is sent back to MAC XYZ and it comes to br0 on VM1. br0 remembered
  that device with MAC XYZ is attached to the same port as eth0, so this
  packet is sent back to eth0, but not to tap01, where it was originated
  from in fact. So network adapter inside Simics will never get DHCP
  reply from dhcpd...

  Workaround for this issue:
  1) $ brctl setageing br0 0 # disables MAC learning on br0
  2) Manually disable hairpin_mode for VM ports: echo 0 > /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode

  However, it's not acceptable for our environment to disable MAC
  learning on br0, so we commented out strings in openstack sources,
  that enables hairpin_mode.

  As far as I understand hairpin_mode is only needed to allow VM ping
  itself with floating ip address. However, we have no floating address
  assigned to that VM. So I suggest to control hairpin_mode for VM ports
  in nova.conf:

  hairpin_mode = 1 # by default

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1264308/+subscriptions