yahoo-eng-team team mailing list archive

[Dan Streetman <dan.streetman@xxxxxxxxxxxxx>]

** Description changed:

  [impact]
  
  in bug 1752391, cloud-init was updated to check if the initramfs has
  specific iscsi root device config (e.g. static initramfs config, or
  ibft), and if so then set up the interface in e/n/i but do not mark it
  as 'auto'.  This prevents ifupdown from actually running dhclient for
  the interface; the reason for the change in that bug was to prevent a
  hang at shutdown, as taking down the interface removes access to the
  iscsi root disk (thus hanging the system).
  
  However, iBFT has no mechanism in its spec to provide the DNS search
  domain, so while the interface ip address/netmask and gateway is
  correctly set up when the system finishes booting, it does not contain
  any search domain in /etc/resolv.conf.  There is no way to get this
  information except from an actual dhcp query/response.
  
  In the initramfs, if open-iscsi does not detect ibft (or have static
  config built-in) then it will not create a /run/net-IFACE.conf file; if
  it does find ibft (or have static config) it will create a /run/net-
  IFACE.conf file with the config data that it has (e.g. with ibft, all
  the interface configuration that ibft is able and configured to
  provide).  After this (setting up the interface and mounting the root
  dev), ipconfig runs and checks for the existence of /run/net-IFACE.conf
  file.  If it does NOT exist, then ipconfig creates it containing the
  result of the dhcp lease response that ipconfig received - which
  includes the dns search domain.  However, if there is already a /run
  /net-IFACE.conf file, ipconfig does NOT update it.  So, the file created
  by open-iscsi remains, without any dns search domain.
  
  Once the system boots, resolvconf checks for existance of the /etc/net-
  IFACE.conf file, and if found it transfers some of the data into
  /etc/resolv.conf (e.g. the dns search domain).  Since it does not
  contain any dns search domain, the /etc/resolv.conf file does not
  contain it either.
  
  If the system is configured to start dhclient for the interface, then
  dhclient receives and sets up the system resolv.conf to use the dns
  search domain correctly.
  
  For trusty: cloud-init is very different and sets up the running system
  ifupdown to start dhclient for the root device interface.  The dns
  search domain is therefore configured by dhclient in the runtime system.
  
  For xenial: cloud-init sets up the iscsi root device's interface with a
  e/n/i configuration for dhcp, but leaves its 'auto' line commented out,
  e.g.:
  
  # control-manual ens3f0
  iface ens3f0 inet dhcp
-     dns-nameservers 169.254.169.254
-     gateway 10.0.0.1
+     dns-nameservers 169.254.169.254
+     gateway 10.0.0.1
  
  this results in no dns search domain for the system.
  
  For bionic and later: cloud-init sets up netplan instead of ifupdown;
- netplan has no such 'auto' mechanism, but it creates systemd-networkd
- .network file with the "CriticalConnection" parameter set to true, which
- prevents systemd-networkd from ever taking the interface down, which
- avoids the hang on shutdown (or any other time dhcp fails/stops for the
- interface).
+ netplan has no such 'auto' mechanism.  The initramfs-tools scripts
+ create a /run/netplan/$DEVICE.yaml file which (when ibft uses dhcp)
+ contains the 'critical' parameter, causing netplan to create a systemd-
+ networkd .network file with the "CriticalConnection" parameter set to
+ true, which prevents systemd-networkd from ever taking the interface
+ down, which avoids the hang on shutdown (or any other time dhcp
+ fails/stops for the interface).  However, it is still possible to remove
+ netplan and install ifupdown in bionic (and later), and when that is
+ done this bug exists, so it should be fixed in those releases also.
  
  [test case]
  
  start a xenial system that uses ibft and check /etc/resolv.conf; there
  will be no dns search domain.
  
+ for bionic or later, start a system that uses ibft, then install
+ ifupdown and resolvconf and remove netplan.io.  When the system is
+ rebooted, the /etc/resolv.conf will not have any dns search domain.
+ 
  [regression potential]
  
- depending on the specific fix, this has the potential to cause the
- interface to be taken down which would then hang the system (due to root
- device being iscsi).
+ as this slightly changes when/how open-iscsi creates a
+ /run/net-$DEVICE.conf file, this has the potential to boot a system with
+ wrong or even no network configuration, when the system configures an
+ iscsi device at boot, e.g. with iBFT.  However, the patch does perform a
+ 'fallback' which should prevent such failure when iBFT does work but
+ ipconfig does not.
  
  [other info]
  
  this appears to have been introduced by bug 1752391.  This bug may
  possibly also exist if the "ip=" parameter is used, which also triggers
  the cloud-init code that sets the runtime interface to 'manual' instead
  of 'auto', though I have not specifically tested that.
+ 
+ Trusty may need to be updated to handle bug 1752391's problem, which
+ would then mean trusty would also need this fix.
+ 
+ Bionic and later do not have this problem with netplan, which is their
+ default, but as noted above if an end-user replaces netplan with
+ ifupdown, this problem still exists.
+ 
+ Bug 1752391 might not be fixed for bionic and later for systems with
+ static iBFT (i.e. not DHCP iBFT) becuase the 'critical' param is only
+ added if DHCP is used (for ipv4 or ipv6).

** Changed in: cloud-init (Ubuntu Cosmic)
       Status: New => Invalid

** Changed in: cloud-init (Ubuntu Xenial)
   Importance: Medium => Undecided

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1806777

Title:
  cloud-init with iscsi ibft keeps interface at 'manual' so the system
  gets no dns search domain

Status in cloud-init:
  Invalid
Status in cloud-init package in Ubuntu:
  Invalid
Status in open-iscsi package in Ubuntu:
  In Progress
Status in cloud-init source package in Trusty:
  Invalid
Status in open-iscsi source package in Trusty:
  New
Status in cloud-init source package in Xenial:
  Invalid
Status in open-iscsi source package in Xenial:
  In Progress
Status in cloud-init source package in Bionic:
  Invalid
Status in open-iscsi source package in Bionic:
  In Progress
Status in cloud-init source package in Cosmic:
  Invalid
Status in open-iscsi source package in Cosmic:
  In Progress
Status in cloud-init source package in Disco:
  Invalid
Status in open-iscsi source package in Disco:
  In Progress

Bug description:
  [impact]

  in bug 1752391, cloud-init was updated to check if the initramfs has
  specific iscsi root device config (e.g. static initramfs config, or
  ibft), and if so then set up the interface in e/n/i but do not mark it
  as 'auto'.  This prevents ifupdown from actually running dhclient for
  the interface; the reason for the change in that bug was to prevent a
  hang at shutdown, as taking down the interface removes access to the
  iscsi root disk (thus hanging the system).

  However, iBFT has no mechanism in its spec to provide the DNS search
  domain, so while the interface ip address/netmask and gateway is
  correctly set up when the system finishes booting, it does not contain
  any search domain in /etc/resolv.conf.  There is no way to get this
  information except from an actual dhcp query/response.

  In the initramfs, if open-iscsi does not detect ibft (or have static
  config built-in) then it will not create a /run/net-IFACE.conf file;
  if it does find ibft (or have static config) it will create a /run
  /net-IFACE.conf file with the config data that it has (e.g. with ibft,
  all the interface configuration that ibft is able and configured to
  provide).  After this (setting up the interface and mounting the root
  dev), ipconfig runs and checks for the existence of /run/net-
  IFACE.conf file.  If it does NOT exist, then ipconfig creates it
  containing the result of the dhcp lease response that ipconfig
  received - which includes the dns search domain.  However, if there is
  already a /run/net-IFACE.conf file, ipconfig does NOT update it.  So,
  the file created by open-iscsi remains, without any dns search domain.

  Once the system boots, resolvconf checks for existance of the /etc
  /net-IFACE.conf file, and if found it transfers some of the data into
  /etc/resolv.conf (e.g. the dns search domain).  Since it does not
  contain any dns search domain, the /etc/resolv.conf file does not
  contain it either.

  If the system is configured to start dhclient for the interface, then
  dhclient receives and sets up the system resolv.conf to use the dns
  search domain correctly.

  For trusty: cloud-init is very different and sets up the running
  system ifupdown to start dhclient for the root device interface.  The
  dns search domain is therefore configured by dhclient in the runtime
  system.

  For xenial: cloud-init sets up the iscsi root device's interface with
  a e/n/i configuration for dhcp, but leaves its 'auto' line commented
  out, e.g.:

  # control-manual ens3f0
  iface ens3f0 inet dhcp
      dns-nameservers 169.254.169.254
      gateway 10.0.0.1

  this results in no dns search domain for the system.

  For bionic and later: cloud-init sets up netplan instead of ifupdown;
  netplan has no such 'auto' mechanism.  The initramfs-tools scripts
  create a /run/netplan/$DEVICE.yaml file which (when ibft uses dhcp)
  contains the 'critical' parameter, causing netplan to create a
  systemd-networkd .network file with the "CriticalConnection" parameter
  set to true, which prevents systemd-networkd from ever taking the
  interface down, which avoids the hang on shutdown (or any other time
  dhcp fails/stops for the interface).  However, it is still possible to
  remove netplan and install ifupdown in bionic (and later), and when
  that is done this bug exists, so it should be fixed in those releases
  also.

  [test case]

  start a xenial system that uses ibft and check /etc/resolv.conf; there
  will be no dns search domain.

  for bionic or later, start a system that uses ibft, then install
  ifupdown and resolvconf and remove netplan.io.  When the system is
  rebooted, the /etc/resolv.conf will not have any dns search domain.

  [regression potential]

  as this slightly changes when/how open-iscsi creates a
  /run/net-$DEVICE.conf file, this has the potential to boot a system
  with wrong or even no network configuration, when the system
  configures an iscsi device at boot, e.g. with iBFT.  However, the
  patch does perform a 'fallback' which should prevent such failure when
  iBFT does work but ipconfig does not.

  [other info]

  this appears to have been introduced by bug 1752391.  This bug may
  possibly also exist if the "ip=" parameter is used, which also
  triggers the cloud-init code that sets the runtime interface to
  'manual' instead of 'auto', though I have not specifically tested
  that.

  Trusty may need to be updated to handle bug 1752391's problem, which
  would then mean trusty would also need this fix.

  Bionic and later do not have this problem with netplan, which is their
  default, but as noted above if an end-user replaces netplan with
  ifupdown, this problem still exists.

  Bug 1752391 might not be fixed for bionic and later for systems with
  static iBFT (i.e. not DHCP iBFT) becuase the 'critical' param is only
  added if DHCP is used (for ipv4 or ipv6).

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1806777/+subscriptions