debcrafters-packages team mailing list archive

[Timo Aaltonen <2100564@xxxxxxxxxxxxxxxxxx>]

An upload of lxd-installer to oracular-proposed has been rejected from
the upload queue for the following reason: "bug marked wontfix".

-- 
You received this bug notification because you are a member of
Debcrafters packages, which is subscribed to lxd-installer in Ubuntu.
https://bugs.launchpad.net/bugs/2100564

Title:
  lxd-installer shim fails to install with snapstore error

Status in lxd-installer package in Ubuntu:
  Fix Released
Status in lxd-installer source package in Focal:
  Won't Fix
Status in lxd-installer source package in Jammy:
  Won't Fix
Status in lxd-installer source package in Noble:
  Won't Fix
Status in lxd-installer source package in Oracular:
  Won't Fix
Status in lxd-installer source package in Plucky:
  Fix Released

Bug description:
  [ Impact ]

  This has been affecting minimal and base pipelines for cloud images
  for both oracular and noble and blocking publication for these images.
  The oracular and noble images fail lxd related tests because lxd
  installation cannot complete with the following error
  `ConnectionResetError: [Errno 104] Connection reset by peer`. This
  error is intermittent.

  The proposed upload allows retries of the installation to mitigate
  installation failures that stem from snapd.

  [ Test Plan ]

  To reproduce the bug you can launch a container from either oracular
  or noble like so: `lxc init ubuntu-minimal-daily:<SUITE> test && lxc
  start test && lxc exec test bash` where
  SUITE=oracular/noble/jammy/focal.

  The error is intermittent and appears to be tied to snapd timing when
  going to standby and the boot time of the target instance being
  tested. To reproduce locally, on the shell connecting to the container
  launched above, you can run the following: `systemctl start snapd ;
  sleep 5 ; lxd init --auto` which produces the following
  `ConnectionResetError: [Errno 104] Connection reset by peer` error.

  The package with the proposed change allows the retry loop to attempt
  installation again after the time window that causes the installation
  failure so this will mitigate the installation errors in our
  pipelines.

  [ Where problems could occur ]

  The possible effect of this change is posting to snapcraft more
  frequently which could increase the traffic to
  `https://api.snapcraft.io`. A discussion with Store follows with a
  confirmation that this possible increase in the number of requests is
  fine from the Store POV.

  [ Other Info ]

  The same change in lxd-installer has been accepted for plucky and is
  has been in build pipelines for our plucky images for a few weeks now;
  these images do not fail on the lxd-installer anymore.

  [Original Description]

  Description:    Ubuntu Plucky Puffin (development branch) (*LXD container)
  Release:        25.04
  Image serial:   20250217
  Source package: https://launchpad.net/ubuntu/plucky/+package/lxd-installer
  Package version:
  ```
  $ apt-cache policy lxd-installer
  lxd-installer:
    Installed: 10
    Candidate: 10
    Version table:
   *** 10 100
          100 /var/lib/dpkg/status
  ```

  Expected behaviour: `lxd init --auto --storage-backend dir` command
  would trigger then `lxd-installer` shim and`lxd` would be installed in
  the container.

  What happened instead:
  ```
  10:19:41 2025-02-17 17:19:41,454 [INFO] test_framework.tests.lxd_start_stop: initializing lxd
  10:19:41 Installing LXD snap, please be patient.
  10:19:41 Traceback (most recent call last):
  10:19:41   File "<string>", line 1, in <module>
  10:19:41     import socket; s=socket.socket(socket.AF_UNIX); s.connect("/run/lxd-installer.socket"); s.send(b"x"); s.recv(1)
  10:21:12                                                                                                           ~~~~~~^^^
  10:21:12 ConnectionResetError: [Errno 104] Connection reset by peer
  ```

  The command that triggers the shim is `lxd init --auto --storage-
  backend dir`

  Reproducer:
  This failure is intermittent, and I have had a hard time getting a reliable reproducer for it. The best that I have got is:
  ```
  lxc init ubuntu-minimal-daily:plucky test #20250217 serial was pulled
  lxc start test
  lxc exec test bash
  <inside the shell>
  lxd init --auto --storage-backend dir # confirm with 'y'
  ```

  The error should appear almost instantly. If it does not, stop &
  delete the instance and repeat the above.

  *LXD team suggested reproducer:
  ```
  $ while lxc launch ubuntu-minimal-daily:plucky test && lxc exec --force-noninteractive test -- lxc --version && lxc delete -f test; do sleep 1; done
  ```

  Logs:
  `journalctl --grep lxd`:
  ```
  root@genuine-satyr:~# journalctl --grep lxd
  Feb 25 23:35:52 genuine-satyr useradd[324]: add 'ubuntu' to group 'lxd'
  Feb 25 23:35:52 genuine-satyr useradd[324]: add 'ubuntu' to shadow group 'lxd'
  Feb 25 23:35:53 genuine-satyr systemd[1]: Starting lxd-installer.socket - Helper to install lxd snap on demand...
  Feb 25 23:35:53 genuine-satyr systemd[1]: Listening on lxd-installer.socket - Helper to install lxd snap on demand.
  Feb 25 23:35:56 genuine-satyr cloud-init[262]: Cloud-init v. 25.1~3geb1965a4-0ubuntu1 finished at Tue, 25 Feb 2025 23:35:56 +0000. Datasource DataSourceLXD.  Up 5.51 seconds
  Feb 25 23:36:00 genuine-satyr systemd[1]: Created slice system-lxd\x2dinstaller.slice - Slice /system/lxd-installer.
  Feb 25 23:36:00 genuine-satyr systemd[1]: Started lxd-installer@0-503-0.service - Helper to install lxd snap on demand (PID 503/UID 0).
  Feb 25 23:36:00 genuine-satyr lxd-installer-service[504]: + [ lxd-installer-service = lxd-installer-service ]
  Feb 25 23:36:00 genuine-satyr lxd-installer-service[511]: + lxd_channel
  Feb 25 23:36:00 genuine-satyr lxd-installer-service[504]: + snap install lxd --channel=5.21/stable/ubuntu-25.04
  Feb 25 23:36:00 genuine-satyr snapd[348]: api_snaps.go:467: Installing snap "lxd" revision unset
  Feb 25 23:36:00 genuine-satyr lxd-installer-service[512]: error: cannot install "lxd": Post "https://api.snapcraft.io/v2/snaps/refresh":
  Feb 25 23:36:00 genuine-satyr systemd[1]: lxd-installer@0-503-0.service: Main process exited, code=exited, status=1/FAILURE
  Feb 25 23:36:00 genuine-satyr systemd[1]: lxd-installer@0-503-0.service: Failed with result 'exit-code'.
  ```

  `journalctl` around failure:
  ```
  Feb 25 23:40:12 test systemd[1]: Created slice system-lxd\x2dinstaller.slice - Slice /system/lxd-installer.
  Feb 25 23:40:12 test systemd[1]: Started lxd-installer@0-508-0.service - Helper to install lxd snap on demand (PID 508/UID 0).
  Feb 25 23:40:12 test lxd-installer-service[509]: + [ lxd-installer-service = lxd-installer-service ]
  Feb 25 23:40:12 test lxd-installer-service[509]: + snap wait system seed.loaded
  Feb 25 23:40:12 test lxd-installer-service[516]: + lxd_channel
  Feb 25 23:40:12 test lxd-installer-service[516]: + track=
  Feb 25 23:40:12 test lxd-installer-service[516]: + [ -r /etc/os-release ]
  Feb 25 23:40:12 test lxd-installer-service[516]: + . /etc/os-release
  Feb 25 23:40:12 test lxd-installer-service[516]: + PRETTY_NAME=Ubuntu Plucky Puffin (development branch)
  Feb 25 23:40:12 test lxd-installer-service[516]: + NAME=Ubuntu
  Feb 25 23:40:12 test lxd-installer-service[516]: + VERSION_ID=25.04
  Feb 25 23:40:12 test lxd-installer-service[516]: + VERSION=25.04 (Plucky Puffin)
  Feb 25 23:40:12 test lxd-installer-service[516]: + VERSION_CODENAME=plucky
  Feb 25 23:40:12 test lxd-installer-service[516]: + ID=ubuntu
  Feb 25 23:40:12 test lxd-installer-service[516]: + ID_LIKE=debian
  Feb 25 23:40:12 test lxd-installer-service[516]: + HOME_URL=https://www.ubuntu.com/
  Feb 25 23:40:12 test lxd-installer-service[516]: + SUPPORT_URL=https://help.ubuntu.com/
  Feb 25 23:40:12 test lxd-installer-service[516]: + BUG_REPORT_URL=https://bugs.launchpad.net/ubuntu/
  Feb 25 23:40:12 test lxd-installer-service[516]: + PRIVACY_POLICY_URL=https://www.ubuntu.com/legal/terms-and-policies/privacy-policy
  Feb 25 23:40:12 test lxd-installer-service[516]: + UBUNTU_CODENAME=plucky
  Feb 25 23:40:12 test lxd-installer-service[516]: + LOGO=ubuntu-logo
  Feb 25 23:40:12 test lxd-installer-service[516]: + track=5.21
  Feb 25 23:40:12 test lxd-installer-service[516]: + [ -n 5.21 ]
  Feb 25 23:40:12 test lxd-installer-service[516]: + [ -n 25.04 ]
  Feb 25 23:40:12 test lxd-installer-service[516]: + echo 5.21/stable/ubuntu-25.04
  Feb 25 23:40:12 test lxd-installer-service[509]: + CHANNEL=5.21/stable/ubuntu-25.04
  Feb 25 23:40:12 test lxd-installer-service[509]: + [ -z 5.21/stable/ubuntu-25.04 ]
  Feb 25 23:40:12 test lxd-installer-service[509]: + snap install lxd --channel=5.21/stable/ubuntu-25.04
  Feb 25 23:40:12 test snapd[345]: api_snaps.go:467: Installing snap "lxd" revision unset
  Feb 25 23:40:12 test snapd[345]: store_download.go:142: no host system xdelta3 available to use deltas
  Feb 25 23:40:12 test lxd-installer-service[517]: error: cannot install "lxd": Post "https://api.snapcraft.io/v2/snaps/refresh":
  Feb 25 23:40:12 test lxd-installer-service[517]:        context canceled
  Feb 25 23:40:12 test systemd[1]: lxd-installer@0-508-0.service: Main process exited, code=exited, status=1/FAILURE
  Feb 25 23:40:12 test systemd[1]: lxd-installer@0-508-0.service: Failed with result 'exit-code'.
  Feb 25 23:40:14 test snapd[345]: overlord.go:518: Released state lock file
  Feb 25 23:40:14 test snapd[345]: daemon stop requested to wait for socket activation
  Feb 25 23:40:14 test systemd[1]: snapd.service: Deactivated successfully.
  Feb 25 23:40:14 test systemd[1]: snapd.service: Consumed 1.014s CPU time, 37.6M memory peak.
  ```

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lxd-installer/+bug/2100564/+subscriptions