canonical-ubuntu-qa team mailing list archive

[Skia <2115731@xxxxxxxxxxxxxxxxxx>]

Okay, here are the facts:

* we have automation running auto-upgrade-testing daily on many images (server, desktop, flavors)
* we run many different upgrade paths: (n->o, o->p, f->j->n, x->b->f->j->n, etc...)
* this automation is running on Jammy, and is based on the `autopkgtest` command, provided by the package in the archive
* until recently it was working with version 5.38ubuntu1~22.04.1
* autopkgtest got SRU'd to version 5.47~22.04.1, and that broke the automation in the following way:
  * the testbed got into a state where it needs a restart before being able to correctly run `do-release-upgrade` (kernel upgrade or else)
  * I don't exactly know what changed to now reach that state, and what made it work before. Probably something around the testbed preparation in `autopkgtest` itself.
  * that make sense and is perfectly fine regarding upgrade testing, it's actually a shortcoming of the auto-upgrade-testing code, which got fixed in the following PR: https://github.com/canonical/auto-upgrade-testing/pull/18
* things are now back in order and working, so that regression actually allowed to improve on upgrade testing. It's just unfortunate that it caught us by surprise.

** Changed in: autopkgtest (Ubuntu Jammy)
       Status: New => Won't Fix

** Changed in: auto-upgrade-testing
       Status: In Progress => Fix Released

** Changed in: autopkgtest (Ubuntu Noble)
       Status: New => Won't Fix

** Changed in: autopkgtest (Ubuntu Oracular)
       Status: New => Won't Fix

** Changed in: autopkgtest (Ubuntu Plucky)
       Status: New => Won't Fix

** Changed in: autopkgtest (Ubuntu Questing)
       Status: New => Won't Fix

** Changed in: autopkgtest (Ubuntu)
       Status: New => Won't Fix

-- 
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to autopkgtest in Ubuntu.
Matching subscriptions: ubuntu-qa-bugs
https://bugs.launchpad.net/bugs/2115731

Title:
  upgrade-testing is broken with autopkgtest >5.47

Status in Auto Upgrade Testing:
  Fix Released
Status in autopkgtest package in Ubuntu:
  Won't Fix
Status in autopkgtest source package in Jammy:
  Won't Fix
Status in autopkgtest source package in Noble:
  Won't Fix
Status in autopkgtest source package in Oracular:
  Won't Fix
Status in autopkgtest source package in Plucky:
  Won't Fix
Status in autopkgtest source package in Questing:
  Won't Fix

Bug description:
  upgrade testing code repo: https://github.com/canonical/auto-upgrade-testing
  upgrade testing profiles repo: https://code.launchpad.net/auto-upgrade-testing-specifications/

  Clone them both side to side, then use the following reproducer:

  ```
  python3 -m upgrade_testing.command_line -c ../auto-upgrade-testing-specifications/profiles/ubuntu-noble-oracular-basic-amd64_qemu.yaml --provision  --adt-args='--timeout-factor=10 -d' --results-dir /tmp/upgrade_tests
  ```

  The upgrade should fail, apparently because now autopkgtest is a bit more eager to reboot when a package needs it before running the tests, and then upgrade-testing checks right away if it's booting on the right system, which is obviously not the case.
  The fix might actually be in upgrade testing code, because all that interaction between autopkgtest and that code has always been very brittle, but I wanted to track the regression in a bug anyway (SRU LP: #2106167).

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-upgrade-testing/+bug/2115731/+subscriptions