canonical-ubuntu-qa team mailing list archive

Thread
Date
[Bug 1691793] Re: Don't allow upstream projects to monopolise resources

To: canonical-ubuntu-qa@xxxxxxxxxxxxxxxxxxx
From: Skia <1691793@xxxxxxxxxxxxxxxxxx>
Date: Mon, 29 Jul 2024 11:44:37 -0000
Reply-to: Bug 1691793 <1691793@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
We implemented some limitation to the amount of running upstream tests:
they're now skipped 2/3 of the time, based on a dice roll when picking
them up. This proportion is easily configurable, but we're currently
happy with the situation. I'm closing this as "Fixed released".

** Changed in: auto-package-testing
       Status: New => Fix Released

-- 
You received this bug notification because you are a member of
Canonical's Ubuntu QA, which is subscribed to Auto Package Testing.
https://bugs.launchpad.net/bugs/1691793

Title:
  Don't allow upstream projects to monopolise resources

Status in Auto Package Testing:
  Fix Released

Bug description:
  At the minute we have a large (> 2000 item) distro queue. The items
  are being processed slowly though, because of the following-

  ubuntu@juju-prod-ues-proposed-migration-machine-3:~> nova list
  +--------------------------------------+------------------------------------------+--------+------------+-------------+-----------------------------------------+
  | ID                                   | Name                                     | Status | Task State | Power State | Networks                                |
  +--------------------------------------+------------------------------------------+--------+------------+-------------+-----------------------------------------+
  | 069799a4-8f67-4b4e-bdf9-24ff30853645 | adt-artful-amd64-snapd-20170518-151451   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.72  |
  | afc56bde-96be-4861-a2e7-b313b6698b7e | adt-artful-amd64-snapd-20170518-152516   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.83  |
  | 8029ed39-2ee8-413b-aa86-a4200ac6d0be | adt-artful-amd64-snapd-20170518-152657   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.85  |
  | c4bdfe21-2506-4169-93c6-52a82ff68185 | adt-artful-amd64-snapd-20170518-154019   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.97  |
  | ab7a818c-0485-42e0-84c2-027664617de4 | adt-artful-i386-binutils-20170518-132701 | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.3   |
  | 97776bda-2652-4c3c-9522-19a2ac8885bf | adt-artful-i386-linux-20170518-153335    | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.82  |
  | 2d043848-1940-445e-88f2-9cf5cc67e89c | adt-xenial-amd64-snapd-20170518-151534   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.73  |
  | bb4137eb-37f1-4c75-9ef6-84bdaba3663f | adt-xenial-amd64-snapd-20170518-153950   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.101 |
  | 5f29277c-d6f6-4046-8047-f4c5b54ba3e7 | adt-xenial-amd64-snapd-20170518-154932   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.98  |
  | c7a1c02f-3bd1-40f6-bc4d-88fa6d6a449c | adt-yakkety-amd64-snapd-20170518-145708  | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.92  |
  | 9908fc7d-1d66-4ef6-9750-05cf147a9206 | adt-yakkety-amd64-snapd-20170518-155016  | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.99  |
  | 8083cf47-b927-4523-b5d0-0172c1b42e22 | adt-yakkety-amd64-snapd-20170518-155058  | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.100 |
  | 6e0a3b49-9f68-4b21-8622-a71d4ca5fffc | adt-yakkety-amd64-snapd-20170518-155416  | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.102 |
  | 8f602c54-9a88-4f6a-90ad-118029bdf526 | adt-zesty-amd64-snapd-20170518-152804    | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.86  |
  | 2e701ea0-658b-4876-a33c-3c4c6a156a47 | adt-zesty-amd64-snapd-20170518-154322    | ACTIVE | -          | Running     | net_ues_proposed_migration=10.42.41.103 |
  +--------------------------------------+------------------------------------------+--------+------------+-------------+-----------------------------------------+
  ubuntu@juju-prod-ues-proposed-migration-machine-3:~> . lgw01.rc 
  ubuntu@juju-prod-ues-proposed-migration-machine-3:~> nova list
  +--------------------------------------+------------------------------------------------------+--------+------------+-------------+------------------------------------------+
  | ID                                   | Name                                                 | Status | Task State | Power State | Networks                                 |
  +--------------------------------------+------------------------------------------------------+--------+------------+-------------+------------------------------------------+
  | d5f2e40f-a760-4928-8e6d-2e6f7f7f7476 | adt-artful-amd64-linux-20170518-113404               | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.126 |
  | 939e8cfe-e321-4835-b335-0d0ba8c5e1f7 | adt-artful-amd64-linux-20170518-150022               | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.110 |
  | a0ae6af7-b87c-4985-904a-edd051fb42e9 | adt-artful-amd64-snapd-20170518-150914               | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.118 |
  | 9cc6a86d-8522-40b4-b96d-6b7aa9c85e8a | adt-artful-i386-libkdegames-20170518-153632          | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.127 |
  | 9f9d742e-d946-4e40-af95-c0ed01582d7e | adt-artful-i386-libkeduvocdocument-20170518-154052   | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.130 |
  | 142c1cdc-edf9-4703-a8c5-c815456c1b31 | adt-artful-i386-r-bioc-annotationdbi-20170518-144326 | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.104 |
  | 4fe944f9-b2d5-4045-b675-b7e8cc4e8375 | adt-xenial-amd64-snapd-20170518-152128               | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.124 |
  | 74b75361-8586-4fdc-a9de-15d8efaef0b7 | adt-xenial-i386-snapd-20170518-134609                | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.94  |
  | bd831c3a-9285-4370-92d8-da871cdc4469 | adt-xenial-i386-snapd-20170518-143152                | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.105 |
  | 609b20a5-1f7d-44e8-b0c1-a9c6b817ea76 | adt-yakkety-amd64-snapd-20170518-150801              | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.119 |
  | 68ba861d-8ad8-48e4-bd1e-b192557d2244 | adt-yakkety-amd64-snapd-20170518-154912              | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.129 |
  | 832f259d-5f0a-4aa5-8b27-761a69bced42 | adt-zesty-amd64-snapd-20170518-140157                | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.100 |
  | 90512bee-10c8-4838-b3e5-8d5c56a00890 | adt-zesty-amd64-snapd-20170518-151441                | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.121 |
  | 680ac016-b6d2-489d-8a9d-6adb07b359c2 | adt-zesty-amd64-snapd-20170518-151922                | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.123 |
  | b9eb663c-ff7b-4dd9-b7e0-770239c508ad | laney-test                                           | ACTIVE | -          | Running     | net_ues_proposed_migration=10.220.44.65  |
  +--------------------------------------+------------------------------------------------------+--------+------------+-------------+------------------------------------------+

  
  snapd is taking up the majority of our quota.

  I think we should consider limiting the amount of concurrent tests any
  one project can run.

To manage notifications about this bug go to:
https://bugs.launchpad.net/auto-package-testing/+bug/1691793/+subscriptions