launchpad-reviewers team mailing list archive

[Roman Yepishev <roman.yepishev@xxxxxxxxx>]

Roman Yepishev has proposed merging lp:~rye/python-oops-tools/local-charset-in-url into lp:python-oops-tools.

Requested reviews:
  Launchpad code reviewers (launchpad-reviewers)
Related bugs:
  Bug #881400 in python-oops-tools: "Loader breaks with DatabaseError when URL contains character outside of ascii range"
  https://bugs.launchpad.net/python-oops-tools/+bug/881400

For more details, see:
https://code.launchpad.net/~rye/python-oops-tools/local-charset-in-url/+merge/80330

Force quoting the URL if the string cannot be presented as ascii during loading.
-- 
https://code.launchpad.net/~rye/python-oops-tools/local-charset-in-url/+merge/80330
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~rye/python-oops-tools/local-charset-in-url into lp:python-oops-tools.

=== modified file 'src/oopstools/oops/models.py'
--- src/oopstools/oops/models.py	2011-10-25 02:25:14 +0000
+++ src/oopstools/oops/models.py	2011-10-25 12:59:23 +0000
@@ -371,6 +371,15 @@
         # We have gotten a ringer, URL's are bytestrings. Encode to UTF8 to get
         # a bytestring and urllib.quote to get a url.
         url = urllib.quote(url.encode('utf8'))
+
+    try:
+        url.encode('ascii')
+    except UnicodeDecodeError:
+        # The URL cannot be represented as ascii but it is not unicode
+        # either. There may be a byte in local encoding and we need to quote
+        # it without any conversion
+        url = urllib.quote(url)
+
     informational = oops.get('informational', 'False').lower() == 'true'
     oops_date = oops.get('time')
     if oops_date is None:

=== modified file 'src/oopstools/oops/test/test_dboopsloader.py'
--- src/oopstools/oops/test/test_dboopsloader.py	2011-10-24 05:21:01 +0000
+++ src/oopstools/oops/test/test_dboopsloader.py	2011-10-25 12:59:23 +0000
@@ -154,3 +154,11 @@
         report = { 'url': 'foo', 'id': 'testnotopichandling'}
         oops = parsed_oops_to_model_oops(report, 'bug_880641')
         self.assertEqual('', oops.pageid)
+
+    def test_broken_url_handling(self):
+        broken_url = '/somep\xe1th'
+        report = { 'url': broken_url, 'id': 'testbrokenurl'}
+        expected_url = urllib.quote(broken_url)
+        oops = parsed_oops_to_model_oops(report, 'test_broken_url_handling')
+        self.assertEqual(expected_url, oops.url)
+