sts-sponsors team mailing list archive

[Matthew Ruffell <1772556@xxxxxxxxxxxxxxxxxx>]

Performing verification of debootstrap using Mauricio's script from
comment #10.

Focal
======

Original
--------

First, using debootstrap from -updates:

$ apt-cache policy debootstrap | grep Installed
Installed: 1.0.118ubuntu1.1

$ tail -n1 *-focal-*.log
==> debootstrap-updates-focal-buildd-http.log <==
I: Base system installed successfully.

==> debootstrap-updates-focal-buildd-https.log <==
E: Couldn't find these debs: apt-transport-https

==> debootstrap-updates-focal--http.log <==
I: Base system installed successfully.

==> debootstrap-updates-focal--https.log <==
E: Couldn't find these debs: apt-transport-https

==> debootstrap-updates-focal-minbase-http.log <==
I: Base system installed successfully.

==> debootstrap-updates-focal-minbase-https.log <==
E: Couldn't find these debs: apt-transport-https

Proposed
--------

I then enabled -proposed and upgraded debootstrap:

$ apt-cache policy debootstrap | grep Installed
Installed: 1.0.118ubuntu1.2

I then re-ran the script.

$ tail -n1 *-proposed-*.log
==> debootstrap-proposed-focal-buildd-http.log <==
I: Base system installed successfully.

==> debootstrap-proposed-focal-buildd-https.log <==
I: Base system installed successfully.

==> debootstrap-proposed-focal--http.log <==
I: Base system installed successfully.

==> debootstrap-proposed-focal--https.log <==
I: Base system installed successfully.

==> debootstrap-proposed-focal-minbase-http.log <==
I: Base system installed successfully.

==> debootstrap-proposed-focal-minbase-https.log <==
I: Base system installed successfully.

Checking the MD5 sums Mauricio checked for HTTP protocol to check same packages
were downloaded:

$ md5sum *-http.log | sort
4421a481335f470fe3cba85f9d56ddfd  debootstrap-proposed-focal-buildd-http.log
4421a481335f470fe3cba85f9d56ddfd  debootstrap-updates-focal-buildd-http.log
828dc970b0c8b3ef523ac0dfd82b6dea  debootstrap-proposed-focal--http.log
828dc970b0c8b3ef523ac0dfd82b6dea  debootstrap-updates-focal--http.log
a3d4733c73de32fb8dd0828eedeedc5d  debootstrap-proposed-focal-minbase-http.log
a3d4733c73de32fb8dd0828eedeedc5d  debootstrap-updates-focal-minbase-http.log

Hashes match.

Checking package diffs between http and https:

DIFF: proposed/https vs. updates/http: variant 'minbase'
--- debootstrap-proposed-focal-minbase-https.log	2020-07-28 17:54:19.793873364 +1200
+++ debootstrap-updates-focal-minbase-http.log	2020-07-28 17:30:15.887212891 +1200
@@ -8 +8 @@
-I: Checking component main on https://mirror.fsmg.org.nz/ubuntu...
+I: Checking component main on http://mirror.fsmg.org.nz/ubuntu...
@@ -21,2 +20,0 @@
-I: Retrieving ca-certificates 20190110ubuntu1
-I: Validating ca-certificates 20190110ubuntu1
@@ -143,2 +140,0 @@
-I: Retrieving libssl1.1 1.1.1f-1ubuntu2
-I: Validating libssl1.1 1.1.1f-1ubuntu2
@@ -175,2 +170,0 @@
-I: Retrieving openssl 1.1.1f-1ubuntu2
-I: Validating openssl 1.1.1f-1ubuntu2
@@ -430 +423,0 @@
-I: Unpacking ca-certificates...
@@ -441 +433,0 @@
-I: Unpacking libssl1.1:amd64...
@@ -445 +436,0 @@
-I: Unpacking openssl...
@@ -449 +439,0 @@
-I: Configuring libssl1.1:amd64...
@@ -460 +449,0 @@
-I: Configuring openssl...
@@ -463 +451,0 @@
-I: Configuring ca-certificates...
@@ -468 +455,0 @@
-I: Configuring ca-certificates...

DIFF: proposed/https vs. updates/http: variant 'buildd'
--- debootstrap-proposed-focal-buildd-https.log	2020-07-28 17:57:58.379585602 +1200
+++ debootstrap-updates-focal-buildd-http.log	2020-07-28 17:33:21.277808831 +1200
@@ -8 +8 @@
-I: Checking component main on https://mirror.fsmg.org.nz/ubuntu...
+I: Checking component main on http://mirror.fsmg.org.nz/ubuntu...
@@ -31,2 +30,0 @@
-I: Retrieving ca-certificates 20190110ubuntu1
-I: Validating ca-certificates 20190110ubuntu1
@@ -211,2 +208,0 @@
-I: Retrieving libssl1.1 1.1.1f-1ubuntu2
-I: Validating libssl1.1 1.1.1f-1ubuntu2
@@ -253,2 +248,0 @@
-I: Retrieving openssl 1.1.1f-1ubuntu2
-I: Validating openssl 1.1.1f-1ubuntu2
@@ -521 +514,0 @@
-I: Unpacking ca-certificates...
@@ -561 +553,0 @@
-I: Unpacking libssl1.1:amd64...
@@ -570 +561,0 @@
-I: Unpacking openssl...
@@ -579 +569,0 @@
-I: Configuring libssl1.1:amd64...
@@ -604 +593,0 @@
-I: Configuring openssl...
@@ -613 +601,0 @@
-I: Configuring ca-certificates...
@@ -640 +627,0 @@
-I: Configuring ca-certificates...

DIFF: proposed/https vs. updates/http: variant ''
--- debootstrap-proposed-focal--https.log	2020-07-28 18:03:50.462303644 +1200
+++ debootstrap-updates-focal--http.log	2020-07-28 17:38:46.136321936 +1200
@@ -8 +8 @@
-I: Checking component main on https://mirror.fsmg.org.nz/ubuntu...
+I: Checking component main on http://mirror.fsmg.org.nz/ubuntu...

Again, only mirror URLs and ssl packages differ.

Based on the test results with debootstrap, the package in -proposed
fixes the problem and doesn't introduce any regressions. I am happy to
mark this as verified.

** Tags removed: verification-needed verification-needed-focal
** Tags added: verification-done-focal

-- 
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1772556

Title:
  d-i netinstall fails due to missing apt-transport-https package

Status in debootstrap package in Ubuntu:
  Confirmed
Status in debootstrap source package in Bionic:
  Fix Committed
Status in debootstrap source package in Eoan:
  Won't Fix
Status in debootstrap source package in Focal:
  Fix Committed
Status in debootstrap package in Debian:
  Fix Released

Bug description:
  [Impact]

  When installing over the network using a netinstall image with pxe
  boot and with an https apt mirror, the installer fails with the error:

    Debootstrap error

    couldn't find these debs: apt-transport-https

    Check /var/log/syslog or see virtual console 4 for the details

  This happens due to apt-transport-https moving to universe from bionic
  onward, but still being required by debootstrap when it sees a https
  apt mirror, even though support for https mirrors is built into apt.

  [Testcase]

  With debootstrap alone:

    $ sudo debootstrap bionic output-dir https://<https-mirror>
    ...

    - Before: "E: Couldn't find these debs: apt-transport-https"
    - After: "I: Base system installed successfully."

  Or with the debian-installer:

  Use the 18.04.4 LTS netinstall ISO to PXE boot with a preseed that
  uses a custom https apt mirror. Something like:

  ```
  d-i mirror/country string manual
  d-i mirror/protocol string https
  d-i mirror/https/hostname string mirrors.ptisp.pt
  d-i mirror/https/directory string /ubuntu/
  d-i mirror/https/proxy string
  ```

  The installer will fail with the error in the impact section.

  There are test packages for debootstrap available in the following
  PPA, for both bionic and focal:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf289200-test

  I have also built a test netinstall ISO with the test debootstrap
  packages, and is available here:

  https://people.canonical.com/~mruffell/sf289200/

  You probably want to use mini.iso for PXE boot, but vmlinuz and
  initrd.gz are provided as well.

  [Regression Potential]

  The fix adds checks for specific distribution releases, so if someone
  is trying to debootstrap a previous release where apt-transport-https
  is still required, it will still function.

  For users of newer releases, it simply omits the package. apt-
  transport-https will still be available in universe if anyone still
  needs it.

  If a regression did occur, users may not be able to access https apt
  mirrors when using debootstrap. In this case, users can use a plain
  http mirror until things are fixed.

  Due to apt-transport-https not being needed in bionic onward, due to
  being built into apt directly, I believe this change won't introduce
  any regressions.

  [Other info]

  The fix landed in upstream debootstrap in the following commit:

  commit 66cbaae642953beba8aec393f3eca076abd89a7d
  From: Hideki Yamane <henrich@xxxxxxxxxx>
  Date: Fri, 28 Feb 2020 00:10:25 +0900
  Subject: select codename for apt-transport-https (Closes: #920255, #879755)
  Link: https://salsa.debian.org/installer-team/debootstrap/-/commit/66cbaae642953beba8aec393f3eca076abd89a7d

  It adds a check for distro release name, and if they fall within Zesty
  and prior, then it requires apt-transport-https, and if Artful and
  later, then it is omitted.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debootstrap/+bug/1772556/+subscriptions