← Back to team overview

curtin-dev team mailing list archive

[Merge] ~raharper/curtin:fix/vmtest-image-sync-after-maas-url-rename into curtin:master

 

Ryan Harper has proposed merging ~raharper/curtin:fix/vmtest-image-sync-after-maas-url-rename into curtin:master.

Commit message:
vmtest: fix image-sync after maas URL stream rename

Image syncing for vmtest has been failing since MAAS renamed their
daily image stream to stable.  In an effort to keep the 'daily' URL
valid the website redirects this to 'stable' path.  However the content
was also modified and includes references to files and URLs which have
the 'stable' name included.  This broke vmtest image sync which expected
the 'daily' string to be present in filenames and content.

The observable result was that curtin would attempt to find content it
had just *synced* to /srv/images but when searching, it would load up
the cached content files which only referenced the 'daily' stream data
and would indicate that the target file was missing.

This branch moves vmtest to using the 'stable' maas image stream
directly.  This value is controllable via an environment variable,
MAAS_IMAGE_STREAM, if set will be used to compose the IMAGE_SRC_URL
and the STREAM_BASE variable which is used to find cached content in
/srv/images/.vmtest_data/ directory.

LP: #1908543


Requested reviews:
  curtin developers (curtin-dev)
Related bugs:
  Bug #1908543 in curtin: "vmtest missing images even after a full vmtest-sync-images run"
  https://bugs.launchpad.net/curtin/+bug/1908543

For more details, see:
https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/396345
-- 
Your team curtin developers is requested to review the proposed merge of ~raharper/curtin:fix/vmtest-image-sync-after-maas-url-rename into curtin:master.
diff --git a/tests/vmtests/image_sync.py b/tests/vmtests/image_sync.py
index e460e02..19b2d22 100644
--- a/tests/vmtests/image_sync.py
+++ b/tests/vmtests/image_sync.py
@@ -32,9 +32,11 @@ def environ_get(key, default):
     return os.environ.get(long_key, os.environ.get(key, default))
 
 
+MAAS_IMAGE_STREAM = environ_get("MAAS_IMAGE_STREAM", "stable")
 IMAGE_SRC_URL = environ_get(
     'IMAGE_SRC_URL',
-    "http://images.maas.io/ephemeral-v3/daily/streams/v1/index.sjson";)
+    "http://images.maas.io/ephemeral-v3/%s/streams/v1/index.sjson"; %
+    MAAS_IMAGE_STREAM)
 IMAGE_DIR = environ_get("IMAGE_DIR", "/srv/images")
 
 KEYRING = environ_get(
@@ -43,7 +45,7 @@ KEYRING = environ_get(
 ITEM_NAME_FILTERS = \
     ['ftype~(boot-initrd|boot-kernel|root-tgz|squashfs)']
 FORMAT_JSON = 'JSON'
-STREAM_BASE = 'com.ubuntu.maas:daily'
+STREAM_BASE = 'com.ubuntu.maas:' + MAAS_IMAGE_STREAM
 VMTEST_CONTENT_ID_PATH_MAP = {
     STREAM_BASE + ":v3:download": "streams/v1/vmtest.json",
     STREAM_BASE + ":centos-bases-download": "streams/v1/vmtest-centos.json",

Follow ups