openlp-core team mailing list archive

[Phill <phill.ridout@xxxxxxxxx>]

Phill has proposed merging lp:~phill-ridout/openlp/1213254_2.0 into lp:openlp/2.0.

Requested reviews:
  Raoul Snyman (raoul-snyman)
Related bugs:
  Bug #1213254 in OpenLP: "OpenLP 1 database importer crashes when encountering control chars"
  https://bugs.launchpad.net/openlp/+bug/1213254

For more details, see:
https://code.launchpad.net/~phill-ridout/openlp/1213254_2.0/+merge/180938

Fix the importing of some OpenLP 1 databases by filtering out invalid xml chars. code from http://stackoverflow.com/questions/8733233/filtering-out-certain-bytes-in-python
-- 
https://code.launchpad.net/~phill-ridout/openlp/1213254_2.0/+merge/180938
Your team OpenLP Core is subscribed to branch lp:openlp/2.0.

=== modified file 'openlp/plugins/songs/lib/xml.py'
--- openlp/plugins/songs/lib/xml.py	2012-12-30 19:41:24 +0000
+++ openlp/plugins/songs/lib/xml.py	2013-08-19 20:19:56 +0000
@@ -78,6 +78,17 @@
 NAMESPACE = u'http://openlyrics.info/namespace/2009/song'
 NSMAP = '{' + NAMESPACE + '}' + '%s'
 
+def clean_xml_string(xml):
+    """
+    Filter out invalid characters in xml
+    Source <http://stackoverflow.com/questions/8733233/filtering-out-certain-bytes-in-python>
+    """
+    return ''.join(char for char in xml if
+        0x20 <= char <= 0xD7FF
+        or char in (0x9, 0xA, 0xD)
+        or 0xE000 <= char <= 0xFFFD
+        or 0x10000 <= char <= 0x10FFFF)
+
 
 class SongXML(object):
     """
@@ -112,6 +123,7 @@
             The verse's language code (ISO-639). This is not required, but
             should be added if available.
         """
+        content = clean_xml_string(content)
         verse = etree.Element(u'verse', type=unicode(type),
             label=unicode(number))
         if lang: