yahoo-eng-team team mailing list archive

[Artom Lifshitz <1899487@xxxxxxxxxxxxxxxxxx>]

While Nova indeed exposes the MTU in our metadata, our source of truth
for that information is Neutron, via the `mtu` field on the Neutron
network.

[As an aside, we've had a long standing issue wherein Neutron allows the
MTU to be mutable, but there's no real support for changing the MTU
within Nova, necessitating the cold migration work around that Frode has
mentioned]

As Neutron controls both the DHCP agent and the `network` API-level
resource, I'm not sure how we'd ever get into a situation where DHCP
provides an MTU value that's different from what's set in the `network`,
but maybe there's a bug in Neutron?

So since Nova's metadata is just a proxy for Neutron's MTU in this case,
I think Brett's question in comment #21 is better asked to the Neutron
folks.

** Project changed: nova => neutron

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1899487

Title:
  cloud-init hard codes MTU configuration at initial deploy time

Status in cloud-init:
  Incomplete
Status in neutron:
  New
Status in netplan.io package in Ubuntu:
  Expired

Bug description:
  When using OpenStack cloud provider `cloud-init` will write out
  /etc/netplan/50-cloud-init.yaml at initial instance boot and not
  update it on subsequent boots of the instance.

  The OpenStack metadata service provides information about MTU for the
  network [0] and cloud-init takes this value and writes it into the
  netplan configuration [1].

  A side effect of configuring the MTU through netplan is that the
  `systemd-networkd` [Link] section [2] gets the MTUBytes value filled
  and this in turn makes `systemd-networkd` ignore the MTU value
  provided by DHCP [3][4].

  During the lifetime of a cloud events occur that will force a operator
  to reduce the MTU available to instances attached to its overlay
  networks. This may happen because of software imposed change of tunnel
  type (GRE -> VXLAN, VXLAN -> GENEVE) or change of topology or
  encapsulation in the physical network equipment.

  To maximize performance these clouds have configured their instances
  to use the maximum available MTU without leaving any headroom to
  account for such changes and the only way to move forward is to reduce
  the available MTU on the instances. We are facing a concrete challenge
  with this now where we have users wanting to migrate from VXLAN
  tunnels to GENEVE tunnels with 38 byte header size.

  0: # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
  {"links": [{"id": "tapa035fb68-01", "vif_id": "a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942, "ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0", "type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id": "b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}

  1: # cat /etc/netplan/50-cloud-init.yaml 
  # This file is generated from information provided by the datasource.  Changes
  # to it will not persist across an instance reboot.  To disable cloud-init's
  # network configuration capabilities, write a file
  # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
  # network: {config: disabled}
  network:
      version: 2
      ethernets:
          ens2:
              dhcp4: true
              match:
                  macaddress: fa:16:3e:31:26:f7
              mtu: 8950
              set-name: ens2

  2: # cat /run/systemd/network/10-netplan-ens2.link 
  [Match]
  MACAddress=fa:16:3e:31:26:f7

  [Link]
  Name=ens2
  WakeOnLan=off
  MTUBytes=8950

  3: # cat /run/systemd/network/10-netplan-ens2.network 
  [Match]
  MACAddress=fa:16:3e:31:26:f7
  Name=ens2

  [Link]
  MTUBytes=8950

  [Network]
  DHCP=ipv4
  LinkLocalAddressing=ipv6

  [DHCP]
  RouteMetric=100
  UseMTU=true

  4: Oct 12 13:30:18 canary-3 systemd-networkd[24084]:
  /run/systemd/network/10-netplan-ens2.network: MTUBytes= in [Link]
  section and UseMTU= in [DHCP] section are set. Disabling UseMTU=.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions