yahoo-eng-team team mailing list archive

[OpenStack Infra <1542108@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/283790
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=4df8d9a7016ab20fce235833d792b89309ec98a7
Submitter: Jenkins
Branch:    master

commit 4df8d9a7016ab20fce235833d792b89309ec98a7
Author: Kevin Benton <kevin@xxxxxxxxxx>
Date:   Mon Feb 22 16:41:45 2016 -0800

    Make agent interface plugging utilize network MTU
    
    This changes the 'plug' and 'plug_new' interfaces of the
    LinuxInterfaceDriver to accept an MTU argument. It then
    updates the dhcp agent and l3 agent to pass the MTU that
    is set on the network that the port belongs to. This allows
    it to take into account the overhead calculations that are
    done for encapsulation types.
    
    It's necessary for the L3 agent to have the MTU because it
    must recognize when fragmentation is needed so it can fragment
    or generate an ICMP error.
    
    It's necessary for the DHCP agent to have the MTU so it doesn't
    interfere when it plugs into a bridge with a larger than 1500
    MTU (the bridge would reduce its MTU to match the agent).
    
    If an operator sets 'network_device_mtu', the value of that
    will be used instead to preserve previous behavior.
    
    Closes-Bug: #1549470
    Closes-Bug: #1542108
    Closes-Bug: #1542475
    DocImpact: Neutron agents now support arbitrary MTU
               configurations on each network (including
               jumbo frames). This is accomplished by checking
               the MTU value defined for each network on which
               it is wiring VIFs.
    Co-Authored-By: Matt Kassawara <mkassawara@xxxxxxxxx>
    Change-Id: Ic091fa78dfd133179c71cbc847bf955a06cb248a


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1542108

Title:
  MTU concerns for the Linux bridge agent

Status in neutron:
  Fix Released

Bug description:
  I ran some experiments with the Linux bridge agent [1] to determine
  the source of MTU problems and offer a potential solution. The
  environment for these experiments contains the following items:

  1) A physical (underlying) network supporting MTU of 1500 or 9000 bytes.
  2) One controller node running the neutron server, Linux bridge agent, L3 agent, DHCP agent, and metadata agent.
  3) One compute node running the Linux bridge agent.
  4) A neutron provider/public network.
  5) A neutron self-service/private network.
  6) A neutron router between the provider and self-service networks.
  7) The self-service network uses the VXLAN protocol with IPv4 endpoints which adds 50 bytes of overhead.
  8) An instance on the self-service network with a floating IP address from an allocation pool on the provider network.

  Background:

  1. For tunnel interfaces, Linux automatically subtracts protocol
  overhead from the parent interface MTU. For example, if eth0 has a
  1500 MTU, a VXLAN interface using it as a parent device has a 1450
  MTU.

  2. For bridge devices, Linux assumes a 1500 MTU and changes the MTU to
  the lowest MTU of any port on the bridge. For example, a bridge
  without ports has a 1500 MTU. If eth0 has a 9000 MTU and you add it as
  a port on the bridge, the bridge changes to a 9000 MTU. If eth1 has a
  1500 MTU and you add it as a port on the bridge, the bridge changes to
  a 1500 MTU.

  3. Only devices that operate at layer-3 can participate in path MTU
  discovery (PMTUD). Therefore, a change of MTU in a layer-2 device such
  as a bridge or veth pair causes that device to discard packets larger
  than the smallest MTU.

  Observations:

  1.  For any physical network MTU, instances must use a MTU value that
  accounts for overlay protocol overhead. Neutron currently offers a way
  to provide a correct value via DHCP. However, it only addresses
  packets outbound from instances. The next two items address packets
  inbound to instances.

  2. For any physical network MTU, each end of the veth pair between the
  self-service network router interface (qr) in the router namespace
  (qrouter) and the self-service network bridge on the controller node
  (qbr) contains a different MTU. The qr end has a 1500 MTU, the default
  value, and the qbr end has a 1450 MTU because the bridge contains a
  VXLAN interface with a 1450 MTU. Thus, the veth pair discards packets
  with a payload larger than 1450 bytes.

  3. For a physical network MTU larger than 1500, each end of the veth
  pair between the provider network router gateway (qg) in the router
  namespace (qrouter) and the provider network bridge on the controller
  node (qbr) contains a different MTU. The qg end has a 1500 MTU, the
  default value, and the qbr end inherits the larger MTU of physical
  network interface. Thus, the veth pair discards packets with a payload
  larger than 1500 bytes.

  Potential solution:

  As per background item (3), MTU disparities must occur in a device
  that operates at layer-3. For example, a router namespace that
  contains interfaces with IP addresses. We can accomplish this task in
  neutron by always using the same MTU on both ends of a veth pair. In
  observation item (2), both ends of the veth pair should use 1450, the
  self-service network MTU. In observation item (3), both ends of the
  veth pair should use 9000, the provider network MTU. If a packet from
  the provider network to the instance has a payload larger than 1450
  bytes, the router can send an ICMP message to the source telling it to
  use a 1450 MTU.

  [1] http://lists.openstack.org/pipermail/openstack-
  dev/2016-January/084241.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1542108/+subscriptions