canonical-ubuntu-qa team mailing list archive

[Jeremy Bícha <2035101@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Example 1
=========
Yesterday, I noticed that gnome-metronome 1.3.0-0ubuntu2 was allowed to migrate from mantic-proposed to mantic despite its own autopkgtest failing on every architecture. This migration happened after an automatic migration-reference run happened which returned the result "neutral". "neutral" is expected for superficial autopkgtests like the one in gnome-metronome. Before that migration-reference run, the failures were correctly shown on the excuses page.

Screenshots attached.

https://autopkgtest.ubuntu.com/packages/gnome-metronome/mantic/amd64

(I then uploaded 1.3.0-0ubuntu3 which fixed the autopkgtest regression)

Here's the gnome-metronome autopkgtest code:
https://salsa.debian.org/a-wai/gnome-metronome/-/blob/debian/master/debian/tests/control

Example 2
=========
https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#webkit2gtk

does not show that the 2.41 series of webkit2gtk triggered an
autopkgtest failure for devhelp

https://autopkgtest.ubuntu.com/packages/devhelp/mantic/amd64

However, Debian's britney is showing the failure:
https://release.debian.org/britney/pseudo-excuses-experimental.html#webkit2gtk

Here's the devhelp autopkgetst code:
https://salsa.debian.org/gnome-team/devhelp/-/blob/debian/latest/debian/tests/control

Theory
======
My theory is that failures are being ignored for packages which only have superficial autopkgtests, like for devhelp and gnome-metronome.

Possibly, this issue has been fixed in Debian's copy of britney already

** Affects: auto-package-testing
     Importance: Undecided
         Status: New

** Attachment added: "Screenshot from 2023-09-10 12-41-01.png"
   https://bugs.launchpad.net/bugs/2035101/+attachment/5699928/+files/Screenshot%20from%202023-09-10%2012-41-01.png

-- 
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to Auto Package Testing.
https://bugs.launchpad.net/bugs/2035101

Title:
  autopkgtest failures are ignored, apparently for packages with only
  superficial tests

Status in Auto Package Testing:
  New

Bug description:
  Example 1
  =========
  Yesterday, I noticed that gnome-metronome 1.3.0-0ubuntu2 was allowed to migrate from mantic-proposed to mantic despite its own autopkgtest failing on every architecture. This migration happened after an automatic migration-reference run happened which returned the result "neutral". "neutral" is expected for superficial autopkgtests like the one in gnome-metronome. Before that migration-reference run, the failures were correctly shown on the excuses page.

  Screenshots attached.

  https://autopkgtest.ubuntu.com/packages/gnome-metronome/mantic/amd64

  (I then uploaded 1.3.0-0ubuntu3 which fixed the autopkgtest
  regression)

  Here's the gnome-metronome autopkgtest code:
  https://salsa.debian.org/a-wai/gnome-metronome/-/blob/debian/master/debian/tests/control

  Example 2
  =========
  https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#webkit2gtk

  does not show that the 2.41 series of webkit2gtk triggered an
  autopkgtest failure for devhelp

  https://autopkgtest.ubuntu.com/packages/devhelp/mantic/amd64

  However, Debian's britney is showing the failure:
  https://release.debian.org/britney/pseudo-excuses-experimental.html#webkit2gtk

  Here's the devhelp autopkgetst code:
  https://salsa.debian.org/gnome-team/devhelp/-/blob/debian/latest/debian/tests/control

  Theory
  ======
  My theory is that failures are being ignored for packages which only have superficial autopkgtests, like for devhelp and gnome-metronome.

  Possibly, this issue has been fixed in Debian's copy of britney
  already

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/2035101/+subscriptions