cloud-init-dev team mailing list archive

[Scott Moser <smoser@xxxxxxxxxx>]

Scott Moser has proposed merging lp:~smoser/cloud-init/trunk.1532072 into lp:cloud-init.

Commit message:
user_data: fix error when user-data is not utf-8 decodable

when user-data was not decodable, cloud-init would raise exception.
This also changes the signature of user_data.convert_string.
The 'headers' argument was never used, and woudl have been broken
if it was, as it was expected to be a dictionary but then was
passed in with *headers.

Requested reviews:
  cloud init development team (cloud-init-dev)
Related bugs:
  Bug #1532072 in cloud-init (Ubuntu): "cloud-init fails with UnicodeDecodeError"
  https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1532072

For more details, see:
https://code.launchpad.net/~smoser/cloud-init/trunk.1532072/+merge/298066
-- 
Your team cloud init development team is requested to review the proposed merge of lp:~smoser/cloud-init/trunk.1532072 into lp:cloud-init.

=== modified file 'ChangeLog'
--- ChangeLog	2016-06-21 18:37:23 +0000
+++ ChangeLog	2016-06-21 20:38:10 +0000
@@ -125,6 +125,7 @@
  - Remove trailing dot from GCE metadata URL (LP: #1581200) [Phil Roche]
  - support network rendering to sysconfig (for centos and RHEL)
  - write_files: if no permissions are given, just use default without warn.
+ - user_data: fix error when user-data is not utf-8 decodable (LP: #1532072)
 
 0.7.6:
  - open 0.7.6

=== modified file 'cloudinit/user_data.py'
--- cloudinit/user_data.py	2016-05-12 17:56:26 +0000
+++ cloudinit/user_data.py	2016-06-21 20:38:10 +0000
@@ -334,19 +334,23 @@
 
 
 # Coverts a raw string into a mime message
-def convert_string(raw_data, headers=None):
+def convert_string(raw_data, content_type=NOT_MULTIPART_TYPE):
     if not raw_data:
         raw_data = ''
-    if not headers:
-        headers = {}
-    data = util.decode_binary(util.decomp_gzip(raw_data))
-    if "mime-version:" in data[0:4096].lower():
-        msg = util.message_from_string(data)
-        for (key, val) in headers.items():
-            _replace_header(msg, key, val)
-    else:
-        mtype = headers.get(CONTENT_TYPE, NOT_MULTIPART_TYPE)
-        maintype, subtype = mtype.split("/", 1)
-        msg = MIMEBase(maintype, subtype, *headers)
+
+    def create_binmsg(data, content_type):
+        maintype, subtype = content_type.split("/", 1)
+        msg = MIMEBase(maintype, subtype)
         msg.set_payload(data)
+        return msg
+
+    try:
+        data = util.decode_binary(util.decomp_gzip(raw_data))
+        if "mime-version:" in data[0:4096].lower():
+            msg = util.message_from_string(data)
+        else:
+            msg = create_binmsg(data, content_type)
+    except UnicodeDecodeError:
+        msg = create_binmsg(raw_data, content_type)
+
     return msg

=== modified file 'tests/unittests/test_data.py'
--- tests/unittests/test_data.py	2016-05-12 20:43:11 +0000
+++ tests/unittests/test_data.py	2016-06-21 20:38:10 +0000
@@ -557,3 +557,15 @@
         ud_proc = ud.UserDataProcessor(self.getCloudPaths())
         message = ud_proc.process(msg)
         self.assertTrue(count_messages(message) == 1)
+
+
+class TestConvert(helpers.TestCase):
+    def test_handles_binary(self):
+        blob = b'\x32\x99'
+        msg = ud.convert_string(blob)
+        self.assertEqual(blob, msg.get_payload(decode=True))
+
+    def test_handle_headers(self):
+        text = "hi mom"
+        msg = ud.convert_string(text)
+        self.assertEqual(text, msg.get_payload(decode=False))