← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #86750

[Bug 1938344] [NEW] Setting snap-store-assertions in a model config causes cloud-init to fail

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

When the juju model config includes a `snap-store-assertions`, the
cloud-init.service runs into problem when trying to contact snapd's
socket.

In this setup, cloud-ctrl01 runs a Juju controller in a LXD container
and cloud-vm02 is a VM created by MAAS. Here is how to reproduce the
issue:

ubuntu@cloud-ctrl01:~$ juju model-config
...
snap-store-assertions         model    |-
  type: account-key
  authority-id: canonical
  revision: 2
...
snap-store-proxy              model    dI5E5ZV6U3wOc919eLmZ0MtOxAyxxTIP
snap-store-proxy-url          default  ""
...

ubuntu@cloud-ctrl01:~$ juju add-machine
created machine 2

ubuntu@cloud-ctrl01:~$ juju ssh 2
...

Inside "machine 2", cloud-init's journal output:

ubuntu@cloud-vm02:~$ journalctl -u cloud-init.service | cat
-- Logs begin at Wed 2021-07-28 21:06:26 UTC, end at Wed 2021-07-28 21:17:01 UTC. --
Jul 28 21:06:31 ubuntu systemd[1]: Starting Initial cloud-init job (metadata service crawler)...
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: Cloud-init v. 21.2-3-g899bfaa9-0ubuntu2~20.04.1 running 'init' at Wed, 28 Jul 2021 21:06:32 +0000. Up 10.23 seconds.
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: | Device |  Up  |           Address           |      Mask     | Scope  |     Hw-Address    |
...
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: +-------+-----------------------+---------------------+-----------+-------+
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: error: cannot assert: cannot communicate with server: Post http://localhost/v2/assertions: dial unix /run/snapd.socket: connect: no such file or directory
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: error: cannot communicate with server: Put http://localhost/v2/snaps/core/conf: dial unix /run/snapd.socket: connect: no such file or directory
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: 2021-07-28 21:06:33,810 - util.py[WARNING]: Failed to run bootcmd module bootcmd
Jul 28 21:06:33 cloud-vm02 cloud-init[893]: 2021-07-28 21:06:33,822 - util.py[WARNING]: Running module bootcmd (<module 'cloudinit.config.cc_bootcmd' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_bootcmd.py'>) failed
Jul 28 21:06:34 cloud-vm02 useradd[989]: new group: name=ubuntu, GID=1000
...
Jul 28 21:06:35 cloud-vm02 systemd[1]: cloud-init.service: Main process exited, code=exited, status=1/FAILURE
Jul 28 21:06:35 cloud-vm02 systemd[1]: cloud-init.service: Failed with result 'exit-code'.
Jul 28 21:06:35 cloud-vm02 systemd[1]: Failed to start Initial cloud-init job (metadata service crawler).

Removing the snap-store-assertions/snap-store-proxy configs from the
model make cloud-init work again.

FYI, my naive attempt at adding "After=sockets.target" to the cloud-
init.service didn't work :/

Additional information:

ubuntu@cloud-vm02:~$ /var/lib/juju/tools/machine-2/jujud version
2.9.9-ubuntu-amd64

$ lsb_release -rd
Description:	Ubuntu 20.04.2 LTS
Release:	20.04

ubuntu@cloud-vm02:~$ apt-cache policy cloud-init
cloud-init:
  Installed: 21.2-3-g899bfaa9-0ubuntu2~20.04.1
  Candidate: 21.2-3-g899bfaa9-0ubuntu2~20.04.1
  Version table:
 *** 21.2-3-g899bfaa9-0ubuntu2~20.04.1 500
        500 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     20.1-10-g71af48df-0ubuntu5 500
        500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages

ubuntu@cloud-ctrl01:~$ juju version
2.9.9-ubuntu-amd64

** Affects: cloud-init
     Importance: Undecided
         Status: New

** Affects: juju
     Importance: Undecided
         Status: New

** Also affects: cloud-init
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1938344

Title:
  Setting snap-store-assertions in a model config causes cloud-init to
  fail

Status in cloud-init:
  New
Status in juju:
  New

Bug description:
  When the juju model config includes a `snap-store-assertions`, the
  cloud-init.service runs into problem when trying to contact snapd's
  socket.

  In this setup, cloud-ctrl01 runs a Juju controller in a LXD container
  and cloud-vm02 is a VM created by MAAS. Here is how to reproduce the
  issue:

  ubuntu@cloud-ctrl01:~$ juju model-config
  ...
  snap-store-assertions         model    |-
    type: account-key
    authority-id: canonical
    revision: 2
  ...
  snap-store-proxy              model    dI5E5ZV6U3wOc919eLmZ0MtOxAyxxTIP
  snap-store-proxy-url          default  ""
  ...

  ubuntu@cloud-ctrl01:~$ juju add-machine
  created machine 2

  ubuntu@cloud-ctrl01:~$ juju ssh 2
  ...

  Inside "machine 2", cloud-init's journal output:

  ubuntu@cloud-vm02:~$ journalctl -u cloud-init.service | cat
  -- Logs begin at Wed 2021-07-28 21:06:26 UTC, end at Wed 2021-07-28 21:17:01 UTC. --
  Jul 28 21:06:31 ubuntu systemd[1]: Starting Initial cloud-init job (metadata service crawler)...
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: Cloud-init v. 21.2-3-g899bfaa9-0ubuntu2~20.04.1 running 'init' at Wed, 28 Jul 2021 21:06:32 +0000. Up 10.23 seconds.
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: ++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: +--------+------+-----------------------------+---------------+--------+-------------------+
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: | Device |  Up  |           Address           |      Mask     | Scope  |     Hw-Address    |
  ...
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: ci-info: +-------+-----------------------+---------------------+-----------+-------+
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: error: cannot assert: cannot communicate with server: Post http://localhost/v2/assertions: dial unix /run/snapd.socket: connect: no such file or directory
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: error: cannot communicate with server: Put http://localhost/v2/snaps/core/conf: dial unix /run/snapd.socket: connect: no such file or directory
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: 2021-07-28 21:06:33,810 - util.py[WARNING]: Failed to run bootcmd module bootcmd
  Jul 28 21:06:33 cloud-vm02 cloud-init[893]: 2021-07-28 21:06:33,822 - util.py[WARNING]: Running module bootcmd (<module 'cloudinit.config.cc_bootcmd' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_bootcmd.py'>) failed
  Jul 28 21:06:34 cloud-vm02 useradd[989]: new group: name=ubuntu, GID=1000
  ...
  Jul 28 21:06:35 cloud-vm02 systemd[1]: cloud-init.service: Main process exited, code=exited, status=1/FAILURE
  Jul 28 21:06:35 cloud-vm02 systemd[1]: cloud-init.service: Failed with result 'exit-code'.
  Jul 28 21:06:35 cloud-vm02 systemd[1]: Failed to start Initial cloud-init job (metadata service crawler).

  Removing the snap-store-assertions/snap-store-proxy configs from the
  model make cloud-init work again.

  FYI, my naive attempt at adding "After=sockets.target" to the cloud-
  init.service didn't work :/

  Additional information:

  ubuntu@cloud-vm02:~$ /var/lib/juju/tools/machine-2/jujud version
  2.9.9-ubuntu-amd64

  $ lsb_release -rd
  Description:	Ubuntu 20.04.2 LTS
  Release:	20.04

  ubuntu@cloud-vm02:~$ apt-cache policy cloud-init
  cloud-init:
    Installed: 21.2-3-g899bfaa9-0ubuntu2~20.04.1
    Candidate: 21.2-3-g899bfaa9-0ubuntu2~20.04.1
    Version table:
   *** 21.2-3-g899bfaa9-0ubuntu2~20.04.1 500
          500 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages
          100 /var/lib/dpkg/status
       20.1-10-g71af48df-0ubuntu5 500
          500 http://us.archive.ubuntu.com/ubuntu focal/main amd64 Packages

  ubuntu@cloud-ctrl01:~$ juju version
  2.9.9-ubuntu-amd64

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1938344/+subscriptions
  Thread Previous • Date Previous • • Thread Next