yahoo-eng-team team mailing list archive

[Jonas Konrad <1979877@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

cloud-init fails to parse network configuration with a KeyError if:

- The configuration has a `nameservers` configuration, and
- The configuration matches an existing interface on the machine, and
- The configuration name does not match that interfaces name.

Consider the following network config:

```
version: 2
ethernets:
  eth:
    match:
      macaddress: '00:11:22:33:44:55'
    addresses: [10.0.0.2/24]
    gateway4: 10.0.0.1
    nameservers: 
      addresses: [10.0.0.1]
```

Now, if a device is present with the mac address 00:11:22:33:44:55,
parsing will fail with the following stack trace:

```
2022-06-25 10:26:43,657 - util.py[WARNING]: failed stage init-local
2022-06-25 10:26:43,657 - util.py[DEBUG]: failed stage init-local
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 738, in status_wrapper
    ret = functor(name, args)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 410, in main_init
    init.apply_network_config(bring_up=bring_up_interfaces)
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 937, in apply_network_config
    return self.distro.apply_network_config(
  File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 231, in apply_network_config
    network_state = parse_net_config_data(netconfig)
  File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 1056, in parse_net_config_data
    nsi.parse_config(skip_broken=skip_broken)
  File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 278, in parse_config
    self.parse_config_v2(skip_broken=skip_broken)
  File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 328, in parse_config_v2
    self._v2_common(command)
  File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 782, in _v2_common
    self._handle_individual_nameserver(name_cmd, iface)
  File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 110, in decorator
    return func(self, command, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 570, in _handle_individual_nameserver
    _iface[iface]["dns"] = {"nameservers": nameservers, "search": search}
KeyError: 'eth'
```

I've investigated the issue myself and created a test case:

```
from unittest import mock

from cloudinit import safeyaml
from cloudinit.net import network_state


class TestNetDns:
    @mock.patch("cloudinit.net.network_state.get_interfaces_by_mac")
    def test_networkd_render_x(self, by_mac):
        by_mac.return_value = {"00:11:22:33:44:55": "foobar"}
        network_state.parse_net_config_data(safeyaml.load("""\
version: 2
ethernets:
  eth:
    match:
      macaddress: '00:11:22:33:44:55'
    addresses: [10.0.0.2/24]
    gateway4: 10.0.0.1
    nameservers: 
      addresses: [10.0.0.1]
"""))
```

This test case will fail with the KeyError.

The reason for this bug is that `handle_ethernets` will take the
interface name from the system because no set-name setting is present.
It will then add the network state under that interface name, not the
configured key 'eth'. Later, _handle_individual_nameserver will try to
look up the state for 'eth' and fail.

A quick bisect shows that this test case starts failing with commit
bf94945fb855c40c5188cef5fb00327c51c41fef (though the mock doesn't work
very far in the past). This commit introduced the name handling logic.

As a workaround, you can give an explicit set-name statement, or change
the key of the network config to match the physical device name.

** Affects: cloud-init
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1979877

Title:
  cloud-init fails with KeyError when nameserver config and matching
  interface are present

Status in cloud-init:
  New

Bug description:
  cloud-init fails to parse network configuration with a KeyError if:

  - The configuration has a `nameservers` configuration, and
  - The configuration matches an existing interface on the machine, and
  - The configuration name does not match that interfaces name.

  Consider the following network config:

  ```
  version: 2
  ethernets:
    eth:
      match:
        macaddress: '00:11:22:33:44:55'
      addresses: [10.0.0.2/24]
      gateway4: 10.0.0.1
      nameservers: 
        addresses: [10.0.0.1]
  ```

  Now, if a device is present with the mac address 00:11:22:33:44:55,
  parsing will fail with the following stack trace:

  ```
  2022-06-25 10:26:43,657 - util.py[WARNING]: failed stage init-local
  2022-06-25 10:26:43,657 - util.py[DEBUG]: failed stage init-local
  Traceback (most recent call last):
    File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 738, in status_wrapper
      ret = functor(name, args)
    File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 410, in main_init
      init.apply_network_config(bring_up=bring_up_interfaces)
    File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 937, in apply_network_config
      return self.distro.apply_network_config(
    File "/usr/lib/python3/dist-packages/cloudinit/distros/__init__.py", line 231, in apply_network_config
      network_state = parse_net_config_data(netconfig)
    File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 1056, in parse_net_config_data
      nsi.parse_config(skip_broken=skip_broken)
    File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 278, in parse_config
      self.parse_config_v2(skip_broken=skip_broken)
    File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 328, in parse_config_v2
      self._v2_common(command)
    File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 782, in _v2_common
      self._handle_individual_nameserver(name_cmd, iface)
    File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 110, in decorator
      return func(self, command, *args, **kwargs)
    File "/usr/lib/python3/dist-packages/cloudinit/net/network_state.py", line 570, in _handle_individual_nameserver
      _iface[iface]["dns"] = {"nameservers": nameservers, "search": search}
  KeyError: 'eth'
  ```

  I've investigated the issue myself and created a test case:

  ```
  from unittest import mock

  from cloudinit import safeyaml
  from cloudinit.net import network_state

  
  class TestNetDns:
      @mock.patch("cloudinit.net.network_state.get_interfaces_by_mac")
      def test_networkd_render_x(self, by_mac):
          by_mac.return_value = {"00:11:22:33:44:55": "foobar"}
          network_state.parse_net_config_data(safeyaml.load("""\
  version: 2
  ethernets:
    eth:
      match:
        macaddress: '00:11:22:33:44:55'
      addresses: [10.0.0.2/24]
      gateway4: 10.0.0.1
      nameservers: 
        addresses: [10.0.0.1]
  """))
  ```

  This test case will fail with the KeyError.

  The reason for this bug is that `handle_ethernets` will take the
  interface name from the system because no set-name setting is present.
  It will then add the network state under that interface name, not the
  configured key 'eth'. Later, _handle_individual_nameserver will try to
  look up the state for 'eth' and fail.

  A quick bisect shows that this test case starts failing with commit
  bf94945fb855c40c5188cef5fb00327c51c41fef (though the mock doesn't work
  very far in the past). This commit introduced the name handling logic.

  As a workaround, you can give an explicit set-name statement, or
  change the key of the network config to match the physical device
  name.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1979877/+subscriptions