← Back to team overview

ubuntu-translations-coordinators team mailing list archive

[Bug 1804478] Re: netplan dhcp interface with no default route causes systemd-networkd-wait-online to hang

 

** Project changed: systemd => ubuntu-translations

** Changed in: ubuntu-translations
   Importance: Unknown => Undecided

** Changed in: ubuntu-translations
       Status: Unknown => New

** Changed in: ubuntu-translations
 Remote watch: github.com/systemd/systemd/issues #3752 => None

** No longer affects: ubuntu-translations

** Changed in: systemd (Ubuntu)
   Importance: Undecided => Medium

** Bug watch removed: github.com/systemd/systemd/issues #3752
   https://github.com/systemd/systemd/issues/3752

-- 
You received this bug notification because you are a member of Ubuntu
Translations Coordinators, which is subscribed to Ubuntu Translations.
Matching subscriptions: Ubuntu Translations bug mail
https://bugs.launchpad.net/bugs/1804478

Title:
  netplan dhcp interface with no default route causes systemd-networkd-
  wait-online to hang

Status in systemd package in Ubuntu:
  Fix Released
Status in systemd source package in Bionic:
  Fix Committed

Bug description:
  [impact]

  systemd-networkd, when configured to use ipv4 dhcp for an interface can hang. This triggers in two known cases:
  a) if configured to ignore the dhcp server's route
  b) the server provides no route
  Then systemd-networkd will hang waiting for the interface's configuration to complete (until it times out).

  This delays boot as well as any restart to systemd-networkd.

  The fix is backporting upstream commit [1]

  [1]: https://github.com/systemd/systemd/commit/223932c7

  [test case]

  There are two ways to test this.
  A) make the system ignore Routes (slightly less realistic but easier to test)
  configure an interface using systemd-networkd:

  $ cat /etc/systemd/network/20-ens7.network
  [Match]
  Name=ens7

  [Network]
  DHCP=ipv4

  [DHCP]
  UseRoutes=false

  then reboot, and check:

  $ systemctl status systemd-networkd-wait-online
  ● systemd-networkd-wait-online.service - Wait for Network to be Configured
     Loaded: loaded (/lib/systemd/system/systemd-networkd-wait-online.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2019-04-08 23:59:26 UTC; 2min 59s ago
  [...]
  Apr 08 23:57:27 lp1804478 systemd[1]: Starting Wait for Network to be Configured...
  Apr 08 23:57:30 lp1804478 systemd-networkd-wait-online[593]: managing: ens3
  Apr 08 23:57:30 lp1804478 systemd-networkd-wait-online[593]: ignoring: lo
  Apr 08 23:59:26 lp1804478 systemd-networkd-wait-online[593]: Event loop failed: Connection timed out
  Apr 08 23:59:26 lp1804478 systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
  Apr 08 23:59:26 lp1804478 systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
  Apr 08 23:59:26 lp1804478 systemd[1]: Failed to start Wait for Network to be Configured.

  B) Make a dhcp server to not provide rules
  Prepare a Ubuntu Bionic Guest under libvirt, e.g. using uvtool:
    $ uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=bionic
    $ uvt-kvm create --password ubuntu bionic arch=amd64 release=bionic label=daily

  Create a isolated network:
  $ cat > isolate.xml << EOF
  <network>
    <name>isolated</name>
    <!-- <bridge name='virbriso1' stp='on' delay='0'/> -->
    <ip address='192.168.251.1' netmask='255.255.255.0'>
      <dhcp>
        <range start='192.168.251.2' end='192.168.251.254'/>
      </dhcp>
    </ip>
  </network>
  EOF
  $ virsh net-define isolate.xml
  $ virsh net-start isolated

  Edit the guest and add that network:
  $ virsh shutdown bionic
  $ virsh edit bionic
  #add this:
    <interface type='network'>
      <source network='isolated'/>
      <mac address='52:54:00:c1:69:08'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </interface>

  In the guest make the device use DHCP:
  Add this to /etc/netplan/50-cloud-init.yaml
          ens7:
              dhcp4: true
              match:
                  macaddress: 52:54:00:c1:69:08
              set-name: ens7
  $ sudo netplan apply

  When rebooting the guest again it will fail
  ● systemd-networkd-wait-online.service - Wait for Network to be Configured
     Loaded: loaded (/lib/systemd/system/systemd-networkd-wait-online.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Tue 2019-04-09 07:07:48 UTC; 1min 41s ago
       Docs: man:systemd-networkd-wait-online.service(8)
    Process: 563 ExecStart=/lib/systemd/systemd-networkd-wait-online (code=exited, status=1/FAILURE)
   Main PID: 563 (code=exited, status=1/FAILURE)

  Apr 09 07:05:48 bionic-dgx2 systemd[1]: Starting Wait for Network to be Configured...
  Apr 09 07:05:50 bionic-dgx2 systemd-networkd-wait-online[563]: managing: ens3
  Apr 09 07:07:48 bionic-dgx2 systemd-networkd-wait-online[563]: Event loop failed: Connection timed out
  Apr 09 07:07:48 bionic-dgx2 systemd[1]: systemd-networkd-wait-online.service: Main process exited, code=exited, status=1/FAILURE
  Apr 09 07:07:48 bionic-dgx2 systemd[1]: systemd-networkd-wait-online.service: Failed with result 'exit-code'.
  Apr 09 07:07:48 bionic-dgx2 systemd[1]: Failed to start Wait for Network to be Configured.

  When working cases A and B will both have the service starting fast and happy.
  $ sudo systemctl status systemd-networkd-wait-online
     Active: active (exited) since Tue 2019-04-09 07:17:16 UTC; 12s ago
  [...]
  Apr 09 07:17:16 bionic-dgx2 systemd-networkd-wait-online[575]: managing: ens3
  [...]
  Apr 09 07:17:16 bionic-dgx2 systemd-networkd-wait-online[575]: managing: ens7

  [regression potential]

  this alters how systemd-networkd works when it starts or restarts,
  specifically how it handles dhcp4; regressions would be around setting
  up interface(s) using dhcpv4, possibly failing to correctly configure
  dhcpv4 interface(s) or even failing internal assertions and exiting.

  The problem is that without a route there will be no netlink even t
  that can be tapped.

  But the change is only a 4 line change that does explicitly
  link->dhcp4_configured = true and calling link_check_ready() in
  dhcp4_address_handler() in case link_set_dhcp_routes() sent no netlink
  messages (dhcp4_messages is zero).

  Therefore the change seems rather small, reviewable and safe.

  [other info]

  original description:

  --

  root cause in systemd bug fixed in Commit
  223932c786ada7f758a7b7878a6ad2dae0d1e5fb:
  https://github.com/systemd/systemd/pull/8728

  Environment:
  Ubuntu 18.04 LTS amd64
  systemd package version 237-3ubuntu10.9

  How to trigger:
  1. add netplan interface with dhcpv4 client enabled:
          enp0s8:
              addresses: []
              dhcp4: true
  2. configure dhcp server to NOT give a default route, or any route for that matter
  3. reboot
  4. systemd-networkd-wait-online will block until dhcp renew is triggered
  root@sensor1:~# /lib/systemd/systemd-networkd-wait-online --timeout=3
  Event loop failed: Connection timed out
  root@sensor1:~# /lib/systemd/systemd-networkd-wait-online --timeout=3 --ignore=enp0s8
  managing: enp0s3
  ignoring: lo
  ignoring: enp0s8

  How to fix:
  Backport upstream changes from systemd (see related systemd ticket).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1804478/+subscriptions