cloud-init-dev team mailing list archive

[Scott Moser <ssmoser2+ubuntu@xxxxxxxxx>]

Scott Moser has proposed merging ~smoser/cloud-init:fix/1768600-utf8-in-user-data into cloud-init:master.

Commit message:
Be more safe on string/bytes when writing multipart user-data to disk.

When creating the multipart mime message that is written as
user-data.txt.i, cloud-init losing data on conversion to some things
as a string.

LP: #1768600

Requested reviews:
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1768600 in cloud-init: "UTF-8 support in User Data (text/x-shellscript) is broken"
  https://bugs.launchpad.net/cloud-init/+bug/1768600

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/+git/cloud-init/+merge/347727

see commit message
-- 
Your team cloud-init commiters is requested to review the proposed merge of ~smoser/cloud-init:fix/1768600-utf8-in-user-data into cloud-init:master.

diff --git a/cloudinit/user_data.py b/cloudinit/user_data.py
index 8f6aba1..ed83d2d 100644
--- a/cloudinit/user_data.py
+++ b/cloudinit/user_data.py
@@ -337,8 +337,10 @@ def is_skippable(part):
 
 # Coverts a raw string into a mime message
 def convert_string(raw_data, content_type=NOT_MULTIPART_TYPE):
+    """convert a string (more likely bytes) or a message into
+    a mime message."""
     if not raw_data:
-        raw_data = ''
+        raw_data = b''
 
     def create_binmsg(data, content_type):
         maintype, subtype = content_type.split("/", 1)
@@ -346,15 +348,17 @@ def convert_string(raw_data, content_type=NOT_MULTIPART_TYPE):
         msg.set_payload(data)
         return msg
 
-    try:
-        data = util.decode_binary(util.decomp_gzip(raw_data))
-        if "mime-version:" in data[0:4096].lower():
-            msg = util.message_from_string(data)
-        else:
-            msg = create_binmsg(data, content_type)
-    except UnicodeDecodeError:
-        msg = create_binmsg(raw_data, content_type)
+    if isinstance(raw_data, six.text_type):
+        bdata = raw_data.encode('utf-8')
+    else:
+        bdata = raw_data
+    bdata = util.decomp_gzip(bdata, decode=False)
+    if b"mime-version:" in bdata[0:4096].lower():
+        msg = util.message_from_string(bdata.decode('utf-8'))
+    else:
+        msg = create_binmsg(bdata, content_type)
 
     return msg
 
+
 # vi: ts=4 expandtab