yahoo-eng-team team mailing list archive

[Chad Smith <1844191@xxxxxxxxxxxxxxxxxx>]

Upstream PR landed with a fix for this issue allowing cloud-init to ignore duplicate macs as seen on  mellanox subordinate devices. 
https://github.com/canonical/cloud-init/pull/1853.

We have also released this into Ubuntu Lunar 23.04 as cloud-init version
22.4-0ubuntu4.

Our plan is also to queue this up as soon as possible for our next SRU
(Stable release update).

Marking this as Fix released as it will be in the next cloud images build for 23.04.
We will create separate bug tasks on this bug for bionic, focal, jammy and kinetic when we start the SRU release process for this bug.

In the meantime, https://code.launchpad.net/~cloud-init-
dev/+archive/ubuntu/daily  has development builds containing this fix
for those looking to validate this behavior before an official SRU
release to Bionic, Focal, jammy and Kinetic.

** Changed in: cloud-init
       Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1844191

Title:
  azure advanced networking sometimes triggers duplicate mac detection

Status in cloud-init:
  Fix Released

Bug description:
  Hi, we're still being affected by this on Azure with
  19.2-24-ge7881d5c-0ubuntu1~18.04.1 - using PACKER to build from image:
  BuildSource : Marketplace/Canonical/UbuntuServer/18.04-DAILY-LTS

  Here is the packer config:
  ````
      "provisioners": [
          {
            "type": "shell",
            "inline": [
              "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done"
            ]
          },
          {
              "type": "ansible",
              "playbook_file": "{{user `ansible_playbook`}}",
              "user": "packer",
              "extra_arguments": [ "--extra-vars", "codeVersion={{user `code_version`}} managed_image_name={{user `managed_image_name`}}" ]
          },
          {
              "type": "shell",
              "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
              "inline_shebang": "/bin/sh -x",
              "inline": [ "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync" ]
      }]
  ````

  Here is the playbook:
  ````
  ---
  - hosts: all
    remote_user: ubuntu
    become: yes
    become_method: sudo
    become_user: root

    environment:
      DEBIAN_FRONTEND: noninteractive
  ````

  Note: we are applying `enableAcceleratedNetworking: true` to the NIC,
  anecdotally we think this is related.

  Usually our playbook has more in it (obviously) but Azure kept
  pointing fingers at us that our image was causing the problem, so I
  ran this test simply deploying a blank deprovisioned image via our
  same process.

  And here's what happens on the serial console log:

  ````
  [   20.337603] sh[910]: + [ -e /var/lib/cloud/instance/obj.pkl ]
  [   20.343177] sh[910]: + echo cleaning persistent cloud-init object
  [   20.349027] [  OK  ] Started Network Time Synchronization.
  [  OK  ] Reached target System Time Synchronized.
  sh[910]: cleaning persistent cloud-init object
  [   20.361066] sh[910]: + rm /var/lib/cloud/instance/obj.pkl
  [   20.412333] sh[910]: + exit 0
  [   34.282291] cloud-init[938]: Cloud-init v. 19.2-24-ge7881d5c-0ubuntu1~18.04.1 running 'init-local' at Mon, 16 Sep 2019 18:02:23 +0000. Up 32.02 seconds.
  [   34.288809] cloud-init[938]: 2019-09-16 18:02:25,262 - util.py[WARNING]: failed stage init-local
  [   34.423057] cloud-init[938]: failed run of stage init-local
  [   34.437716] cloud-init[938]: ------------------------------------------------------------
  [   34.441088] cloud-init[938]: Traceback (most recent call last):
  [   34.443719] cloud-init[938]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 653, in status_wrapper
  [   34.448072] cloud-init[938]:     ret = functor(name, args)
  [   34.450532] cloud-init[938]:   File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 362, in main_init
  [   34.454849] cloud-init[938]:     init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
  [   34.458725] cloud-init[938]:   File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 697, in apply_network_config
  [   34.463421] cloud-init[938]:     net.wait_for_physdevs(netcfg)
  [   34.466051] cloud-init[938]:   File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 344, in wait_for_physdevs
  [   34.470673] cloud-init[938]:     present_macs = get_interfaces_by_mac().keys()
  [   34.473964] cloud-init[938]:   File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 633, in get_interfaces_by_mac
  [   34.479325] cloud-init[938]:     (name, ret[mac], mac))
  [   34.481838] cloud-init[938]: RuntimeError: duplicate mac found! both 'eth0' and 'enP1s1' have mac '00:0d:3a:7c:f7:3f'
  [   34.486614] cloud-init[938]: ------------------------------------------------------------
  [FAILED] Failed to start Initial cloud-init job (pre-networking).
  See 'systemctl status cloud-init-local.service' for details.
  [  OK  ] Reached target Network (Pre).
           Starting Network Service...
  [  OK  ] Started Network Service.
           Starting Wait for Network to be Configured...
           Starting Network Name Resolution...
  [  OK  ] Started Wait for Network to be Configured.
           Starting Initial cloud-init job (metadata service crawler)...
  [  OK  ] Started Network Name Resolution.
  [  OK  ] Reached target Host and Network Name Lookups.
  [  OK  ] Reached target Network.
  ````

  When this happens, the machine never boots, and we get an
  OSProvisioningTimedOut error after about 30 minutes, and the machine
  never reaches healthy state.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1844191/+subscriptions