sts-sponsors team mailing list archive

[Eric Desrochers <eric.desrochers@xxxxxxxxxxxxx>]

** Changed in: containerd (Ubuntu Bionic)
     Assignee: (unassigned) => Jorge Niedbalski (niedbalski)

** Changed in: containerd (Ubuntu Bionic)
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of STS
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1867398

Title:
  [Regression] unsupported protocol scheme

Status in containerd package in Ubuntu:
  Fix Released
Status in containerd source package in Bionic:
  Fix Committed
Status in containerd source package in Eoan:
  Fix Released
Status in containerd source package in Focal:
  Fix Released

Bug description:
  [Description]

  Kubernetes 1.16.17
  Containerd 1.3.3
  Ubuntu Bionic

  [Affected Releases]

   containerd | 1.3.3-0ubuntu1~18.04.1 | bionic-updates/universe  | source, amd64, arm64, armhf, i386, ppc64el, s390x
   containerd | 1.3.3-0ubuntu1~19.10.1 | eoan-updates/universe    | source, amd64, arm64, armhf, i386, ppc64el, s390x
   containerd | 1.3.3-0ubuntu1         | focal                    | source, amd64, arm64, armhf, ppc64el, s390x

  [Impact]

  Reported upstream:
  https://github.com/containerd/containerd/issues/4108

  User Impact:

  Since the Ubuntu bionic-updates bump of the version 1.3.3 through [0] https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1854841
  a regression was introduced.

  The following endpoint description stopped working when scheduling
  pods with k8s 1.16-1.17 isn't longer working.

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."niedbalski-bastion.cloud.sts:5000"]
        endpoint = ["niedbalski-bastion.cloud.sts:5000"]

  As an example, creating a k8s pod defined as following:

  apiVersion: v1
  kind: Pod
  metadata:
    name: busybox
    namespace: default
  spec:
    containers:
      - name: busybox
        image: niedbalski-bastion.cloud.sts:5000/busybox:latest
        command:
          - sleep
          - "3600"
    imagePullSecrets:
      - name: regcred
    restartPolicy: Always

  Will fail in the current Bionic-updates version with the following
  error:

  " failed to do request: Head niedbalski-
  bastion.cloud.sts:///v2/busybox/manifests/latest: unsupported protocol
  scheme "niedbalski-bastion.cloud.sts"

  Normal Scheduled default-scheduler Successfully assigned default/busybox to juju-3a79d2-00268738-4
  Normal Pulling 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Pulling image "niedbalski-bastion.cloud.sts:5000/busybox:latest"
  Warning Failed 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Failed to pull image "niedbalski-bastion.cloud.sts:5000/busybox:latest": rpc error: code = Unknown desc = failed to pull and unpack image "niedbalski-bastion.cloud.sts:5000/busybox:latest": failed to resolve reference "niedbalski-bastion.cloud.sts:5000/busybox:latest": failed to do request: Head niedbalski-bastion.cloud.sts:///v2/busybox/manifests/latest: unsupported protocol scheme "niedbalski-bastion.cloud.sts"
  Warning Failed 8m39s (x4 over 10m) kubelet, juju-3a79d2-00268738-4 Error: ErrImagePull
  Warning Failed 8m27s (x6 over 10m) kubelet, juju-3a79d2-00268738-4 Error: ImagePullBackOff
  Normal BackOff 4m56s (x21 over 10m) kubelet, juju-3a79d2-00268738-4 Back-off pulling image "niedbalski-bastion.cloud.sts:5000/busybox:latest"

  [Test Case]

  1) Configure a private docker repository repository

  2)  Modify the containerd registry mirror config as follows:
  ** http://paste.ubuntu.com/p/yP63WMkVT6/

  3) Execute the following pod (http://paste.ubuntu.com/p/BVYQFMfCmk/)

  Status of the scheduled pod should be ImagePullBackOff
  and the before mentioned error should be raised.

  [Possible workaround and solution]

  As a workaround change the endpoint to support the scheme (https://)
  Provide a fallback mechanism for URL parsing validation to fallback to http or https.
  I suspect that this change introduced on 1.3.3 through
  0b29c9c) may be the offending commit.

  [Regression Potential]

  ** The change proposed on the SRU takes in consideration both cases
  1) a endpoint without a schema 2) a endpoint with a schema.

  1) worked in 1.2.6 as explained in the "Impact section" and stopped
  being supported with the current Bionic version 1.3.3, 2) Should work
  on both cases.

  In neither case this should break existing endpoint definitions
  now new deployments of containerd.

  [Other Info]

  ** This commit upstream
  https://github.com/containerd/containerd/commit/a022c218194c05449ad69b69c48fc6cac9d6f0b3
  addresses the issue.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/containerd/+bug/1867398/+subscriptions