canonical-ubuntu-qa team mailing list archive
-
canonical-ubuntu-qa team
-
Mailing list archive
-
Message #02484
[Bug 2035101] Re: autopkgtest failures are ignored, apparently for packages with only superficial tests
superficial tests that pass are marked as neutral. In Debian's
implementation, this means that a package that only has superficial
autopkgtests will NOT get the autopkgtest "bounty" which reduces the
number of days a package must wait in Unstable before automatically
migrating to Testing. (The bounty is set to 3 so a typical upload will
take 2 days instead of 5 days).
https://salsa.debian.org/ci-
team/autopkgtest/-/blob/master/doc/README.package-tests.rst
https://salsa.debian.org/release-
team/britney2/-/blob/master/etc/britney.conf#L81
However, if a superficial autopkgtest fails, as happens with libwacom
2.9.0-1 , it must be treated as a failure.
A migration-reference/0 of neutral should not allow a failing
autopkgtest to migrate.
As of today, libwacom is still not fixed and is only held in proposed by
my block-proposed bug so I think this is a clear test case right now.
--
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to Auto Package Testing.
https://bugs.launchpad.net/bugs/2035101
Title:
autopkgtest failures are ignored, apparently for packages with only
superficial tests
Status in Auto Package Testing:
New
Status in britney:
New
Bug description:
Example 1
=========
Yesterday, I noticed that gnome-metronome 1.3.0-0ubuntu2 was allowed to migrate from mantic-proposed to mantic despite its own autopkgtest failing on every architecture. This migration happened after an automatic migration-reference run happened which returned the result "neutral". "neutral" is expected for superficial autopkgtests like the one in gnome-metronome. Before that migration-reference run, the failures were correctly shown on the excuses page.
Screenshots attached.
https://autopkgtest.ubuntu.com/packages/gnome-metronome/mantic/amd64
(I then uploaded 1.3.0-0ubuntu3 which fixed the autopkgtest
regression)
Here's the gnome-metronome autopkgtest code:
https://salsa.debian.org/a-wai/gnome-metronome/-/blob/debian/master/debian/tests/control
Example 2
=========
https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#webkit2gtk
does not show that the 2.41 series of webkit2gtk triggered an
autopkgtest failure for devhelp
https://autopkgtest.ubuntu.com/packages/devhelp/mantic/amd64
However, Debian's britney is showing the failure:
https://release.debian.org/britney/pseudo-excuses-experimental.html#webkit2gtk
Here's the devhelp autopkgetst code:
https://salsa.debian.org/gnome-team/devhelp/-/blob/debian/latest/debian/tests/control
Theory
======
My theory is that failures are being ignored for packages which only have superficial autopkgtests, like for devhelp and gnome-metronome.
Possibly, this issue has been fixed in Debian's copy of britney
already
To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/2035101/+subscriptions
References