openlp-core team mailing list archive

[Phill <phill.ridout@xxxxxxxxx>]

Phill has proposed merging lp:~phill-ridout/openlp/bug1251437_2.0 into lp:openlp/2.0.

Requested reviews:
  OpenLP Core (openlp-core)
Related bugs:
  Bug #1251437 in OpenLP: "BibleGateway importer crashes on non unicode urls"
  https://bugs.launchpad.net/openlp/+bug/1251437

For more details, see:
https://code.launchpad.net/~phill-ridout/openlp/bug1251437_2.0/+merge/198147

It appears that when a url encoded in unicode is requested with urllib2.urlopen .getUrl() returns unicode. However the url that we request in the biblegateway importer contains redirects. In this case the redirect uses an utf-8 url (im guessing that) urllib2 takes this as an utf-8 encoded url, so returns getUrl() encoded as utf-8.
-- 
https://code.launchpad.net/~phill-ridout/openlp/bug1251437_2.0/+merge/198147
Your team OpenLP Core is requested to review the proposed merge of lp:~phill-ridout/openlp/bug1251437_2.0 into lp:openlp/2.0.

=== modified file 'openlp/core/utils/__init__.py'
--- openlp/core/utils/__init__.py	2013-08-17 08:51:08 +0000
+++ openlp/core/utils/__init__.py	2013-12-07 09:27:48 +0000
@@ -464,7 +464,11 @@
     log.debug(u'Downloading URL = %s' % url)
     try:
         page = urllib2.urlopen(req)
-        log.debug(u'Downloaded URL = %s' % page.geturl())
+        downloaded_url = page.geturl()
+        # Sometimes we get redirected, in this case page.geturl is encoded in utf-8
+        if not isinstance(downloaded_url, unicode):
+            downloaded_url = downloaded_url.decode('utf-8')
+        log.debug(u'Downloaded URL = %s' % downloaded_url)
     except urllib2.URLError:
         log.exception(u'The web page could not be downloaded')
     if not page: