canonical-ubuntu-qa team mailing list archive

[Paride Legovini <2067087@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

There are cases were we had autopkgtest run against the wrong version of
the trigger package; britney can be confused by this and potentially
migrate untested packages. In practice we hit this issue in a couple of
cases:

- We were [unnecessarily] running the setup-testbed script as a setup
command (autopkgtest --setup-commands), on top of running it at image
creation time. One thing the script does is clearing the
/etc/apt/sources.list.d/* sources. With the switch to deb822 sources,
this caused the -proposed source to be lost, so we were always pulling
and testing packages from the release pocket.

- We happened to pass the wrong image to autopkgtest. The autopkgtest
tool just worked fine, but of course tested packages from the wrong
release.

These issues are not strictly speaking src:autopkgtest bugs: they are at
the boundary between bugs in autopkgtest-cloud and user errors. However
ideally autopkgtest should check that the desired trigger versions are
used in the test.

Possible ways to implement this:

- When using the --apt-pocket=pocket[=pkgname,src:srcname,...] syntax,
we could make autopkgtest verify that pkgname (if installed) actually
comes from the specified pocket. This should not be problematic to
implement, but won't cover some cases (like: we pass the wrong image,
and the trigger package also happens to have a version in -proposed for
that release).

- We extent the --apt-pocket=pocket[=pkgname,src:srcname,...] to support
passing a version number, e.g. --apt-
pocket=pocket[=pkgname/VERSION,src:srcname/VERSION,...], with fallback
to the old behavior if /VERSION is not specified. Then we teach
autopkgtest-cloud to pass /VERSION. On the autopkgtest side, it may be
possible to implement this by only using pins, so with little ad-hoc
logic. The src:srcname/VERSION is tricky, because the src package and
bin package versions may be different. However newer apt versions
support pinning by source package, we could use that functionality. This
way we'd cover fairly recent release and the devel release, which is
were automatic migrations happen.

** Affects: auto-package-testing
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to Auto Package Testing.
https://bugs.launchpad.net/bugs/2067087

Title:
  There is no mechanism preventing the wrong trigger package version
  from being tested

Status in Auto Package Testing:
  New

Bug description:
  There are cases were we had autopkgtest run against the wrong version
  of the trigger package; britney can be confused by this and
  potentially migrate untested packages. In practice we hit this issue
  in a couple of cases:

  - We were [unnecessarily] running the setup-testbed script as a setup
  command (autopkgtest --setup-commands), on top of running it at image
  creation time. One thing the script does is clearing the
  /etc/apt/sources.list.d/* sources. With the switch to deb822 sources,
  this caused the -proposed source to be lost, so we were always pulling
  and testing packages from the release pocket.

  - We happened to pass the wrong image to autopkgtest. The autopkgtest
  tool just worked fine, but of course tested packages from the wrong
  release.

  These issues are not strictly speaking src:autopkgtest bugs: they are
  at the boundary between bugs in autopkgtest-cloud and user errors.
  However ideally autopkgtest should check that the desired trigger
  versions are used in the test.

  Possible ways to implement this:

  - When using the --apt-pocket=pocket[=pkgname,src:srcname,...] syntax,
  we could make autopkgtest verify that pkgname (if installed) actually
  comes from the specified pocket. This should not be problematic to
  implement, but won't cover some cases (like: we pass the wrong image,
  and the trigger package also happens to have a version in -proposed
  for that release).

  - We extent the --apt-pocket=pocket[=pkgname,src:srcname,...] to
  support passing a version number, e.g. --apt-
  pocket=pocket[=pkgname/VERSION,src:srcname/VERSION,...], with fallback
  to the old behavior if /VERSION is not specified. Then we teach
  autopkgtest-cloud to pass /VERSION. On the autopkgtest side, it may be
  possible to implement this by only using pins, so with little ad-hoc
  logic. The src:srcname/VERSION is tricky, because the src package and
  bin package versions may be different. However newer apt versions
  support pinning by source package, we could use that functionality.
  This way we'd cover fairly recent release and the devel release, which
  is were automatic migrations happen.

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/2067087/+subscriptions