canonical-ubuntu-qa team mailing list archive

[Tim Andersson <mp+488920@xxxxxxxxxxxxxxxxxx>]

Tim Andersson has proposed merging utah:internal-server-errors into utah:master.

Requested reviews:
  UTAH Dev (utah)

For more details, see:
https://code.launchpad.net/~utah/utah/+git/utah/+merge/488920

add retries to internal urlretrieve (to get an internal file) to minimise flakiness.
-- 
Your team UTAH Dev is requested to review the proposed merge of utah:internal-server-errors into utah:master.

diff --git a/utah/provisioning/provisioning.py b/utah/provisioning/provisioning.py
index 23dd7d4..c33b2b2 100644
--- a/utah/provisioning/provisioning.py
+++ b/utah/provisioning/provisioning.py
@@ -27,6 +27,7 @@ import re
 import shutil
 import stat
 import subprocess
+import time
 import sys
 import urllib
 import urllib.parse
@@ -159,8 +160,19 @@ class Machine(object):
                 # TODO: implement download utility function and use it here
                 if "://" not in path:
                     path = "file://" + path
-                filename = urllib.request.urlretrieve(path,
-                                              reporthook=self.dldisplay)[0]
+                # hacky little workaround - this sporadically tempfails, so let's just
+                # add some retries!
+                filename = None
+                for _ in range(20):
+                    try:
+                        filename = urllib.request.urlretrieve(path,
+                                                    reporthook=self.dldisplay)[0]
+                        break
+                    except urllib.error.HTTPError:
+                        time.sleep(3)
+                if not filename:
+                    raise urllib.error.HTTPError(f"Failed to retrieve {path}")
+
                 setattr(self, item, filename)
                 self.logger.debug('%s is locally available as %s',
                                   path, getattr(self, item))