canonical-ubuntu-qa team mailing list archive

[Tim Andersson <mp+463761@xxxxxxxxxxxxxxxxxx>]

Tim Andersson has proposed merging ~andersson123/autopkgtest-cloud:admin-page-running-for-logtail-mismatch-heuristic into autopkgtest-cloud:master.

Requested reviews:
  Canonical's Ubuntu QA (canonical-ubuntu-qa)
Related bugs:
  Bug #2058463 in Auto Package Testing: "Admin page should also display tests with mismatched running for and timestamp in logtail"
  https://bugs.launchpad.net/auto-package-testing/+bug/2058463

For more details, see:
https://code.launchpad.net/~andersson123/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/463761
-- 
Your team Canonical's Ubuntu QA is requested to review the proposed merge of ~andersson123/autopkgtest-cloud:admin-page-running-for-logtail-mismatch-heuristic into autopkgtest-cloud:master.

diff --git a/charms/focal/autopkgtest-web/webcontrol/helpers/admin.py b/charms/focal/autopkgtest-web/webcontrol/helpers/admin.py
index afc406b..a717f7b 100644
--- a/charms/focal/autopkgtest-web/webcontrol/helpers/admin.py
+++ b/charms/focal/autopkgtest-web/webcontrol/helpers/admin.py
@@ -1,14 +1,25 @@
+import re
+
 MAX_DURATION_FACTOR_BEFORE_CONSIDERED_WEIRD = 5
+MIN_LOGTAIL_DEVIATION = 0.75
+TIMESTAMP_REGEX = "^[0-9]+s"
 
 
 def select_abnormally_long_jobs(running_info, get_test_id, db_con):
     global MAX_DURATION_FACTOR_BEFORE_CONSIDERED_WEIRD
+    filter_re = re.compile(TIMESTAMP_REGEX.encode("UTF-8"), re.DOTALL)
     pruned_running_info = {}
     for running_pkg, running_dict in running_info.items():
         for skey, sval in running_dict.items():
             for release, values in sval.items():
                 for arch, vals in values.items():
                     duration = vals[1]
+                    logtail = vals[2]
+                    timestamp = logtail.split(" ")[0]
+                    timestamps_deviation = 1
+                    if filter_re.search(timestamp):
+                        logtail_running_for = int(timestamp.replace("s", ""))
+                        timestamps_deviation = logtail_running_for / duration
                     test_id = get_test_id(release, arch, running_pkg)
                     if test_id is None:
                         continue
@@ -21,7 +32,7 @@ def select_abnormally_long_jobs(running_info, get_test_id, db_con):
                         (duration)
                         > duration_avg
                         * MAX_DURATION_FACTOR_BEFORE_CONSIDERED_WEIRD
-                    ):
+                    ) or (timestamps_deviation < MIN_LOGTAIL_DEVIATION):
                         pruned_running_info.setdefault(
                             running_pkg, {}
                         ).setdefault(skey, {}).setdefault(