canonical-ubuntu-qa team mailing list archive

[Tim Andersson <mp+467378@xxxxxxxxxxxxxxxxxx>]

Tim Andersson has proposed merging ~andersson123/autopkgtest-cloud:publish-db-fix into autopkgtest-cloud:master.

Commit message:
fix: web: use ftpmaster.internal for publish-db

This commit modifies the publish-db script to use ftpmaster.internal
instead of archive.ubuntu.com. ftpmaster.internal is obviously a lot
faster and should help improve the speed of the publish-db script.

The script was recently failing hitting various urls at
archive.ubuntu.com, with a ConnectionResetError. This commit also adds
that exception to the try except block which attempts to download the
Sources.gz file for a component/pocket/release combination, just to
stop the script from completely failing if this happens again. As
mentioned, the script was failing completely, meaning the public
database wasn't getting updated at all.

Requested reviews:
  Canonical's Ubuntu QA (canonical-ubuntu-qa)

For more details, see:
https://code.launchpad.net/~andersson123/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/467378

publish-db has been recently failing trying to download Sources.gz files from archive.ubuntu.com. This MP changes publish-db to use ftpmaster.internal instead, and also catches the necessary exception when running into the aforementioned issue - though it may now not be necessary, it'd still help in the future if the same issue did arise.
-- 
Your team Canonical's Ubuntu QA is requested to review the proposed merge of ~andersson123/autopkgtest-cloud:publish-db-fix into autopkgtest-cloud:master.

diff --git a/charms/focal/autopkgtest-web/webcontrol/publish-db b/charms/focal/autopkgtest-web/webcontrol/publish-db
index cfda10f..a621f8a 100755
--- a/charms/focal/autopkgtest-web/webcontrol/publish-db
+++ b/charms/focal/autopkgtest-web/webcontrol/publish-db
@@ -22,7 +22,7 @@ sqlite3.paramstyle = "named"
 config = None
 db_con = None
 
-archive_url = "http://archive.ubuntu.com/ubuntu";
+archive_url = "http://ftpmaster.internal/ubuntu";
 components = ["main", "restricted", "universe", "multiverse"]
 
 
@@ -179,7 +179,7 @@ def get_sources(db_con, release):
                                 },
                             )
                 db_con.commit()
-            except urllib.error.HTTPError as e:
+            except (urllib.error.HTTPError, ConnectionResetError) as e:
                 if e.code == 304:
                     logging.debug("url {} not modified".format(url))