canonical-ubuntu-qa team mailing list archive

[Steve Langasek <2035101@xxxxxxxxxxxxxxxxxx>]

https://ubuntu-archive-team.ubuntu.com/proposed-
migration/log/noble/2024-01-12/23:28:17.log.gz shows that libwacom is in
the 'autopkgtest-pending.json' for amd64 and s390x.  So that's a bug, to
start with.

(We have a bug in general that state/autopkgtest-pending.json does not
get properly garbage-collected, so grows without bound.  File is
currently 112477 lines long.  There are 22,000 results marked as 'Test
in progress' on update_excuses, but it's still excessive?)

A deep dive into https://ubuntu-archive-team.ubuntu.com/proposed-
migration/noble/update_excuses.yaml.xz (which shows information that
update_excuses.html hides, for browser performance reasons) shows:

- excuses:
  - 'Migration status for libwacom (2.8.0-1 to 2.9.0-1): BLOCKED: Rejected/violates
    migration policy/introduces a regression'
  - 'Issues preventing migration:'
  - 'Additional info:'
  - 4 days old
  - Not touching package as requested in <a href="https://launchpad.net/bugs/2049237";>bug
    2049237</a> on Fri Jan 12 22:25:14 2024
  is-candidate: false
  item-name: libwacom
  maintainer: Timo Aaltonen
  migration-policy-verdict: REJECTED_PERMANENTLY
  new-version: 2.9.0-1
  old-version: 2.8.0-1
  policy_info:
    age:
      age-requirement: 0
      current-age: 4.585891203703704
      verdict: PASS
    autopkgtest:
      cinnamon-control-center/5.8.2-1:
        amd64:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/amd64/c/cinnamon-control-center/20240113_163413_fd9be@/log.gz
        - https://autopkgtest.ubuntu.com/packages/c/cinnamon-control-center/noble/amd64
        - null
        - null
        arm64:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/arm64/c/cinnamon-control-center/20240113_084446_18593@/log.gz
        - https://autopkgtest.ubuntu.com/packages/c/cinnamon-control-center/noble/arm64
        - null
        - null
        armhf:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/armhf/c/cinnamon-control-center/20240113_030203_158e7@/log.gz
        - https://autopkgtest.ubuntu.com/packages/c/cinnamon-control-center/noble/armhf
        - null
        - null
        ppc64el:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/ppc64el/c/cinnamon-control-center/20240113_120123_f31b7@/log.gz
        - https://autopkgtest.ubuntu.com/packages/c/cinnamon-control-center/noble/ppc64el
        - null
        - null
        s390x:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/c/cinnamon-control-center/20240112_191145_ac481@/log.gz
        - https://autopkgtest.ubuntu.com/packages/c/cinnamon-control-center/noble/s390x
        - null
        - null
      libinput/1.23.0-2.1:
        amd64:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/amd64/libi/libinput/20240113_163212_eefe2@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libi/libinput/noble/amd64
        - null
        - null
        arm64:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/arm64/libi/libinput/20240113_084221_c4aed@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libi/libinput/noble/arm64
        - null
        - null
        armhf:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/armhf/libi/libinput/20240113_025903_f671b@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libi/libinput/noble/armhf
        - null
        - null
        i386:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/i386/libi/libinput/20240112_191802_f671b@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libi/libinput/noble/i386
        - null
        - null
        ppc64el:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/ppc64el/libi/libinput/20240113_121112_d712f@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libi/libinput/noble/ppc64el
        - null
        - null
        s390x:
        - PASS
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/libi/libinput/20240112_191141_a75f6@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libi/libinput/noble/s390x
        - null
        - null
      libwacom/2.9.0-1:
        amd64:
        - ALWAYSFAIL
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/amd64/libw/libwacom/20240113_163456_db271@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libw/libwacom/noble/amd64
        - null
        - null
        arm64:
        - ALWAYSFAIL
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/arm64/libw/libwacom/20240113_084308_348cc@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libw/libwacom/noble/arm64
        - null
        - null
        armhf:
        - ALWAYSFAIL
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/armhf/libw/libwacom/20240113_025905_0cb24@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libw/libwacom/noble/armhf
        - null
        - null
        i386:
        - ALWAYSFAIL
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/i386/libw/libwacom/20240112_192325_dec21@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libw/libwacom/noble/i386
        - null
        - null
        ppc64el:
        - ALWAYSFAIL
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/ppc64el/libw/libwacom/20240113_121054_348cc@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libw/libwacom/noble/ppc64el
        - null
        - null
        s390x:
        - ALWAYSFAIL
        - https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/s390x/libw/libwacom/20240112_191135_3dcde@/log.gz
        - https://autopkgtest.ubuntu.com/packages/libw/libwacom/noble/s390x
        - null
        - null

[...]

So britney is seeing this as an 'alwaysfail' result, meaning not a
regression.

noble/state/autopkgtest-results.cache correctly labels the libwacom
autopkgtest/2.8.0 results as 'NEUTRAL', not "FAILED".

So the issue is in the merging of the information about the old vs new
states.

-- 
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to Auto Package Testing.
https://bugs.launchpad.net/bugs/2035101

Title:
  autopkgtest failures are ignored, apparently for packages with only
  superficial tests

Status in Auto Package Testing:
  New
Status in britney:
  New

Bug description:
  Example 1
  =========
  Yesterday, I noticed that gnome-metronome 1.3.0-0ubuntu2 was allowed to migrate from mantic-proposed to mantic despite its own autopkgtest failing on every architecture. This migration happened after an automatic migration-reference run happened which returned the result "neutral". "neutral" is expected for superficial autopkgtests like the one in gnome-metronome. Before that migration-reference run, the failures were correctly shown on the excuses page.

  Screenshots attached.

  https://autopkgtest.ubuntu.com/packages/gnome-metronome/mantic/amd64

  (I then uploaded 1.3.0-0ubuntu3 which fixed the autopkgtest
  regression)

  Here's the gnome-metronome autopkgtest code:
  https://salsa.debian.org/a-wai/gnome-metronome/-/blob/debian/master/debian/tests/control

  Example 2
  =========
  https://ubuntu-archive-team.ubuntu.com/proposed-migration/update_excuses.html#webkit2gtk

  does not show that the 2.41 series of webkit2gtk triggered an
  autopkgtest failure for devhelp

  https://autopkgtest.ubuntu.com/packages/devhelp/mantic/amd64

  However, Debian's britney is showing the failure:
  https://release.debian.org/britney/pseudo-excuses-experimental.html#webkit2gtk

  Here's the devhelp autopkgetst code:
  https://salsa.debian.org/gnome-team/devhelp/-/blob/debian/latest/debian/tests/control

  Theory
  ======
  My theory is that failures are being ignored for packages which only have superficial autopkgtests, like for devhelp and gnome-metronome.

  Possibly, this issue has been fixed in Debian's copy of britney
  already

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/2035101/+subscriptions