yahoo-eng-team team mailing list archive

[Sean Dague <sean@xxxxxxxxx>]

** Changed in: nova
       Status: Incomplete => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1264308

Title:
  hairpin_mode should be configurable in config file

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  I'm running Havana multinode (Neutron LB plugin). By default Openstack
  sets hairpin_mode for VM ports connected to bridge:

  # cat /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode
  1

  With this flag set broadcast packets are reflected back from bridge to
  interface, where they came from. That's do not cause any problems
  until you have bridges inside VM.

  Network diagram:
  VM1  [SIMICS] -> [tap01] -> [br0] -> [eth0] ->
  HOST -> [tap6cfe8817-2a] -> [brqd1485e0f-cc] -> [tapXXX] ->
  VM2  -> [eth0] -> [dhcpd]

  Simics (http://en.wikipedia.org/wiki/Simics) simulates Linux box with
  internal network interface attached to tap01. Usually we have more
  than one taps. DHCP client inside Simics requests IP address from DHCP
  server running on VM2.

  DHCP request packet with MAC address XYZ of network adapter that
  exists within Simics instance is reflected back from [brqd1485e0f-cc]
  to [tap6cfe8817-2a]. When it reaches br0 again, br0 rewrites own MAC
  learning table: packet is came from eth0 side, so device with MAC
  address XYZ is attached to the same port as eth0.

  Anyway original DHCP request reaches dhcpd running on VM2. DHCP reply
  is sent back to MAC XYZ and it comes to br0 on VM1. br0 remembered
  that device with MAC XYZ is attached to the same port as eth0, so this
  packet is sent back to eth0, but not to tap01, where it was originated
  from in fact. So network adapter inside Simics will never get DHCP
  reply from dhcpd...

  Workaround for this issue:
  1) $ brctl setageing br0 0 # disables MAC learning on br0
  2) Manually disable hairpin_mode for VM ports: echo 0 > /sys/class/net/brqd1485e0f-cc/brif/tap6cfe8817-2a/hairpin_mode

  However, it's not acceptable for our environment to disable MAC
  learning on br0, so we commented out strings in openstack sources,
  that enables hairpin_mode.

  As far as I understand hairpin_mode is only needed to allow VM ping
  itself with floating ip address. However, we have no floating address
  assigned to that VM. So I suggest to control hairpin_mode for VM ports
  in nova.conf:

  hairpin_mode = 1 # by default

  ** tcpdumps

  $ tcpdump -ni eth0
  ...
  14:35:51.354925 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from XX:YY:ac:19:01:00, length: 359 // original DHCP request
  14:35:51.355083 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from XX:YY:ac:19:01:00, length: 359 // reflected DHCP request
  14:35:52.312025 IP 9.6.24.224.bootps > 9.6.24.150.bootpc: BOOTP/DHCP, Reply, length: 300 // DHCP reply reached eth0

  $ tcpdump -ni tap01
  tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  listening on tap01, link-type EN10MB (Ethernet), capture size 96 bytes
  14:40:46.533975 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from XX:YY:ac:19:01:00, length: 359 // original DHCP request
  14:40:46.534182 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from XX:YY:ac:19:01:00, length: 359 // reflected DHCP request
  // no DHCP reply captured

  ** br0 MAC learning table

  $ brctl showmacs br0
  port no	mac addr		is local?	ageing timer
    1	XX:YY:ac:19:01:00	no		  93.34 // VM1 simics MAC - should be on port 2, not 1
    2	c6:a1:84:46:8e:de	yes		   0.00 // VM1 tap01 MAC
    3	ca:9f:6b:b3:ce:ca	yes		   0.00 // VM1 tap02 MAC
    1	fa:16:3e:1b:76:32	yes		   0.00 // VM1 eth0 MAC
    1	fa:16:3e:9a:c0:45	no		  91.34 // VM2 eth0 MAC

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1264308/+subscriptions