launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #24409
[Merge] ~cjwatson/launchpad:remove-xpi-importer into launchpad:master
Colin Watson has proposed merging ~cjwatson/launchpad:remove-xpi-importer into launchpad:master.
Commit message:
Remove XPI import support
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
Related bugs:
Bug #1016336 in Launchpad itself: "old_xmlplus is in sourcecode"
https://bugs.launchpad.net/launchpad/+bug/1016336
For more details, see:
https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380059
As far as I can tell, this hasn't been used on production since about 2011, possibly due to changes in the Firefox release model. It's sufficiently complicated that it isn't worth keeping if it isn't being used, although we can reintroduce it later if necessary.
The initial motivation for this was that it seems surprisingly difficult to parse DTDs in modern Python without the non-Python-3-friendly copy of parts of the old python-xml package that we've been keeping around in sourcecode/old_xmlplus, but it turned into a larger opportunity to prune unused code.
I had to start by rearranging test_xpi_po_exporter to avoid using the XPI importer, as it makes some sense to preserve vestigial support for exporting XPI templates in PO format in order that there's some way to extract existing information from the production database.
--
Your team Launchpad code reviewers is requested to review the proposed merge of ~cjwatson/launchpad:remove-xpi-importer into launchpad:master.
diff --git a/lib/lp/translations/configure.zcml b/lib/lp/translations/configure.zcml
index 9f91581..3693a4b 100644
--- a/lib/lp/translations/configure.zcml
+++ b/lib/lp/translations/configure.zcml
@@ -291,15 +291,6 @@
<allow
interface="lp.translations.interfaces.translationimporter.ITranslationFormatImporter"/>
</class>
- <class
- class="lp.translations.utilities.mozilla_xpi_importer.MozillaXpiImporter">
- <allow
- interface="lp.translations.interfaces.translationimporter.ITranslationFormatImporter"/>
- </class>
- <subscriber
- for="lp.translations.interfaces.translationimportqueue.ITranslationImportQueueEntry"
- provides="lp.translations.interfaces.translationimporter.ITranslationFormatImporter"
- factory="lp.translations.utilities.mozilla_xpi_importer.MozillaXpiImporter"/>
<!-- PO File -->
diff --git a/lib/lp/translations/doc/poexport-language-pack.txt b/lib/lp/translations/doc/poexport-language-pack.txt
index 94b3624..062abc8 100644
--- a/lib/lp/translations/doc/poexport-language-pack.txt
+++ b/lib/lp/translations/doc/poexport-language-pack.txt
@@ -33,9 +33,6 @@ This is handy for examining the tar files that are generated.
... if not member.isreg():
... # Not a regular file. No size to print.
... size = '-'
- ... elif member.name.endswith('.xpi'):
- ... # XPI file. Binary, so don't try counting lines.
- ... size = 'bin'
... else:
... size = len(tarfile.extractfile(member).readlines())
... print("| %5s | %s" % (size, member.name))
@@ -102,15 +99,17 @@ And one of the included .po files look like what we expected.
'# traducci\xc3\xb3n de es.po al Spanish\n'
-Language pack with XPI translations
------------------------------------
+Base language pack export using Librarian with date limits
+----------------------------------------------------------
-Launchpad supports XPI file imports. However, we don't have an export
-process ready, and thus, we do it with an external script that does that
-last part based on .po files and the original en-US.xpi file. To achieve
-that, we export all translations for XPI files in a special directory:
-xpi/translation_domain/
+Launchpad is also able to generate a tarball of all files for a
+distribution series that only includes translation files which have been
+changed since a certain date.
+First we need to set up some data to test with, and for this we need
+some DB classes.
+
+ >>> from StringIO import StringIO
>>> from lp.registry.interfaces.distribution import IDistributionSet
>>> from lp.registry.interfaces.person import IPersonSet
>>> from lp.registry.model.sourcepackagename import SourcePackageName
@@ -126,122 +125,6 @@ Get the Grumpy distro series.
>>> series = getUtility(IDistributionSet)['ubuntu'].getSeries('grumpy')
-
-Sample data initialization
-..........................
-
-We need to import an XPI template and a translation to see those files
-exported as part of language packs.
-
- >>> from lp.translations.enums import RosettaImportStatus
- >>> from lp.translations.interfaces.translationimportqueue import (
- ... ITranslationImportQueue)
- >>> from lp.translations.utilities.tests.test_xpi_import \
- ... import get_en_US_xpi_file_to_import
-
-We are going to import translations for mozilla-firefox package in
-grumpy distro series.
-
- >>> series = getUtility(IDistributionSet)['ubuntu'].getSeries('grumpy')
- >>> spn = SourcePackageName.byName('mozilla-firefox')
- >>> pot_header = 'Content-Type: text/plain; charset=UTF-8\n'
- >>> firefox_template = POTemplate(
- ... name='firefox', translation_domain='firefox',
- ... distroseries=series, sourcepackagename=spn,
- ... owner=mark, languagepack=True, path='en-US.xpi',
- ... header=pot_header)
-
-Attach the en-US.xpi (the template) file.
-
- >>> en_US_xpi = get_en_US_xpi_file_to_import('en-US')
- >>> translation_import_queue = getUtility(ITranslationImportQueue)
- >>> by_maintainer = True
- >>> template_entry = translation_import_queue.addOrUpdateEntry(
- ... firefox_template.path, en_US_xpi, by_maintainer,
- ... mark, distroseries=series, sourcepackagename=spn,
- ... potemplate=firefox_template)
-
-Attach the es.xpi file (the translation) file.
-
- >>> es_xpi = get_en_US_xpi_file_to_import('en-US')
- >>> firefox_es_translation = firefox_template.newPOFile('es')
- >>> translation_entry = translation_import_queue.addOrUpdateEntry(
- ... 'es.xpi', es_xpi, by_maintainer,
- ... mark, distroseries=series, sourcepackagename=spn,
- ... potemplate=firefox_template,
- ... pofile=firefox_es_translation)
-
-Before we are ready to import the attached files, we need to approve
-them first.
-
- >>> template_entry.setStatus(
- ... RosettaImportStatus.APPROVED, rosetta_experts)
- >>> translation_entry.setStatus(
- ... RosettaImportStatus.APPROVED, rosetta_experts)
-
-Given that the files are attached to Librarian, we need to commit the
-transaction to make sure it's stored properly and available.
-
- >>> transaction.commit()
-
-We do now the import from the queue:
-
- >>> (subject, body) = firefox_template.importFromQueue(template_entry)
- >>> (subject, body) = firefox_es_translation.importFromQueue(
- ... translation_entry)
- >>> flush_database_caches()
- >>> transaction.commit()
-
-
-Language pack export with XPI files
-...................................
-
-We are now ready to get an exported language pack with XPI files.
-
- >>> language_pack = export_language_pack(
- ... distribution_name='ubuntu',
- ... series_name='grumpy',
- ... component=None,
- ... force_utf8=True,
- ... output_file=None,
- ... logger=logger)
- >>> transaction.commit()
-
-We get other entries in language pack + en-US.xpi file and the
-translations in .po file format.
-
- >>> tarfile = bytes_to_tarfile(language_pack.file.read())
- >>> examine_tarfile(tarfile)
- | - | rosetta-grumpy
- | 1 | rosetta-grumpy/mapping.txt
- | 1 | rosetta-grumpy/timestamp.txt
- | - | rosetta-grumpy/xpi
- | - | rosetta-grumpy/xpi/firefox
- | bin | rosetta-grumpy/xpi/firefox/en-US.xpi
- | 94 | rosetta-grumpy/xpi/firefox/en.po
- | 102 | rosetta-grumpy/xpi/firefox/es.po
-
-We got a valid en-US.xpi file.
-
- >>> fh = tarfile.extractfile('rosetta-grumpy/xpi/firefox/en-US.xpi')
- >>> from zipfile import ZipFile
- >>> zip = ZipFile(fh, 'r')
- >>> sorted(zip.namelist())
- ['chrome.manifest', 'chrome/en-US.jar', 'copyover3.png', 'install.rdf']
-
-
-Base language pack export using Librarian with date limits
-----------------------------------------------------------
-
-Launchpad is also able to generate a tarball of all files for a
-distribution series that only includes translation files which have been
-changed since a certain date.
-
-First we need to set up some data to test with, and for this we need
-some DB classes.
-
- >>> from StringIO import StringIO
-
Get a source package name to go with our distro series.
>>> spn = SourcePackageName.byName('evolution')
@@ -319,12 +202,9 @@ Check that the log looks ok.
>>> print(logger.getLogBuffer())
DEBUG Selecting PO files for export
- INFO Number of PO files to export: 4
- DEBUG Exporting PO file ... (1/4)
- DEBUG Exporting PO file ... (2/4)
- DEBUG Exporting PO file ... (3/4)
- DEBUG Exporting PO file ... (4/4)
- INFO Exporting XPI template files.
+ INFO Number of PO files to export: 2
+ DEBUG Exporting PO file ... (1/2)
+ DEBUG Exporting PO file ... (2/2)
INFO Adding timestamp file
INFO Adding mapping file
INFO Done.
@@ -342,13 +222,8 @@ Check that the log looks ok.
| - | rosetta-grumpy/es
| - | rosetta-grumpy/es/LC_MESSAGES
| 21 | rosetta-grumpy/es/LC_MESSAGES/test.po
- | 2 | rosetta-grumpy/mapping.txt
+ | 1 | rosetta-grumpy/mapping.txt
| 1 | rosetta-grumpy/timestamp.txt
- | - | rosetta-grumpy/xpi
- | - | rosetta-grumpy/xpi/firefox
- | bin | rosetta-grumpy/xpi/firefox/en-US.xpi
- | 94 | rosetta-grumpy/xpi/firefox/en.po
- | 102 | rosetta-grumpy/xpi/firefox/es.po
Check the files look OK.
@@ -431,21 +306,15 @@ should get only files that were updated after 2000-01-02.
>>> tarfile = bytes_to_tarfile(language_pack.file.read())
Now, there is only one file exported for the 'test' domain, the one that
-had the modification date after the last generated language pack. We
-ignore the xpi entries because those are outside the scope of this test.
+had the modification date after the last generated language pack.
>>> examine_tarfile(tarfile)
| - | rosetta-grumpy
| - | rosetta-grumpy/cy
| - | rosetta-grumpy/cy/LC_MESSAGES
| 21 | rosetta-grumpy/cy/LC_MESSAGES/test.po
- | 2 | rosetta-grumpy/mapping.txt
+ | 1 | rosetta-grumpy/mapping.txt
| 1 | rosetta-grumpy/timestamp.txt
- | - | rosetta-grumpy/xpi
- | - | rosetta-grumpy/xpi/firefox
- | bin | rosetta-grumpy/xpi/firefox/en-US.xpi
- | 94 | rosetta-grumpy/xpi/firefox/en.po
- | 102 | rosetta-grumpy/xpi/firefox/es.po
There is another situation where a translation file is exported again as
part of a language pack update, even without being changed. It is re-
@@ -488,13 +357,8 @@ template has. That's why we get both translations:
| - | rosetta-grumpy/es
| - | rosetta-grumpy/es/LC_MESSAGES
| 21 | rosetta-grumpy/es/LC_MESSAGES/test.po
- | 2 | rosetta-grumpy/mapping.txt
+ | 1 | rosetta-grumpy/mapping.txt
| 1 | rosetta-grumpy/timestamp.txt
- | - | rosetta-grumpy/xpi
- | - | rosetta-grumpy/xpi/firefox
- | bin | rosetta-grumpy/xpi/firefox/en-US.xpi
- | 94 | rosetta-grumpy/xpi/firefox/en.po
- | 102 | rosetta-grumpy/xpi/firefox/es.po
Script arguments and concurrency
@@ -543,7 +407,6 @@ different distribution and series combinations.
/var/lock/launchpad-language-pack-exporter__ubuntu__hoary.lock
INFO Exporting translations for series hoary of distribution ubuntu.
INFO Number of PO files to export: 12
- INFO Exporting XPI template files.
INFO Adding timestamp file
INFO Adding mapping file
INFO Done.
diff --git a/lib/lp/translations/doc/sourcepackagerelease-translations.tar.gz b/lib/lp/translations/doc/sourcepackagerelease-translations.tar.gz
index 4473202..5a26175 100644
Binary files a/lib/lp/translations/doc/sourcepackagerelease-translations.tar.gz and b/lib/lp/translations/doc/sourcepackagerelease-translations.tar.gz differ
diff --git a/lib/lp/translations/doc/sourcepackagerelease-translations.txt b/lib/lp/translations/doc/sourcepackagerelease-translations.txt
index 039d644..0a03aff 100644
--- a/lib/lp/translations/doc/sourcepackagerelease-translations.txt
+++ b/lib/lp/translations/doc/sourcepackagerelease-translations.txt
@@ -83,11 +83,10 @@ And the queue should have 2 entries, with exactly the same contents.
>>> queue_entries = translation_import_queue.getAllEntries(target=sp_test)
>>> queue_entries.count()
- 2
+ 1
>>> for entry in queue_entries:
... print(entry.path, entry.importer.name)
- something/en-US.xpi maria
po/es.po maria
Commit, so the uploaded translations become available to the scripts.
diff --git a/lib/lp/translations/scripts/language_pack.py b/lib/lp/translations/scripts/language_pack.py
index 34d28c7..77fe80a 100644
--- a/lib/lp/translations/scripts/language_pack.py
+++ b/lib/lp/translations/scripts/language_pack.py
@@ -32,9 +32,6 @@ from lp.services.librarian.interfaces.client import (
from lp.services.tarfile_helpers import LaunchpadWriteTarFile
from lp.translations.enums import LanguagePackType
from lp.translations.interfaces.languagepack import ILanguagePackSet
-from lp.translations.interfaces.translationfileformat import (
- TranslationFileFormat,
- )
from lp.translations.interfaces.vpoexport import IVPOExportSet
@@ -93,7 +90,6 @@ def export(distroseries, component, update, force_utf8, logger):
# XXX JeroenVermeulen 2008-02-06: Is there anything here that we can unify
# with the export-queue code?
- xpi_templates_to_export = set()
path_prefix = 'rosetta-%s' % distroseries.name
pofiles = export_set.get_distroseries_pofiles(
@@ -137,14 +133,7 @@ def export(distroseries, component, update, force_utf8, logger):
domain = potemplate.translation_domain.encode('ascii')
code = pofile.getFullLanguageCode().encode('UTF-8')
-
- if potemplate.source_file_format == TranslationFileFormat.XPI:
- xpi_templates_to_export.add(potemplate)
- path = os.path.join(
- path_prefix, 'xpi', domain, '%s.po' % code)
- else:
- path = os.path.join(
- path_prefix, code, 'LC_MESSAGES', '%s.po' % domain)
+ path = os.path.join(path_prefix, code, 'LC_MESSAGES', '%s.po' % domain)
try:
# We don't want obsolete entries here, it makes no sense for a
@@ -160,19 +149,6 @@ def export(distroseries, component, update, force_utf8, logger):
store.invalidate(pofile)
- logger.info("Exporting XPI template files.")
- librarian_client = getUtility(ILibrarianClient)
- for template in xpi_templates_to_export:
- if template.source_file is None:
- logger.warning(
- "%s doesn't have source file registered." % potemplate.title)
- continue
- domain = template.translation_domain.encode('ascii')
- archive.add_file(
- os.path.join(path_prefix, 'xpi', domain, 'en-US.xpi'),
- librarian_client.getFileByAlias(
- template.source_file.id).read())
-
logger.info("Adding timestamp file")
# Is important that the timestamp contain the date when the export
# started, not when it finished because that notes how old is the
diff --git a/lib/lp/translations/scripts/tests/test_validate_translations_file.py b/lib/lp/translations/scripts/tests/test_validate_translations_file.py
index 8311718..4f1fe28 100644
--- a/lib/lp/translations/scripts/tests/test_validate_translations_file.py
+++ b/lib/lp/translations/scripts/tests/test_validate_translations_file.py
@@ -16,9 +16,6 @@ from lp.translations.scripts.validate_translations_file import (
UnknownFileType,
ValidateTranslationsFile,
)
-from lp.translations.utilities.tests.xpi_helpers import (
- get_en_US_xpi_file_to_import,
- )
class TestValidateTranslationsFile(TestCase):
@@ -45,30 +42,6 @@ class TestValidateTranslationsFile(TestCase):
self.assertRaises(
UnknownFileType, validator._validateContent, 'foo.bar', 'content')
- def test_validate_dtd_good(self):
- validator = self._makeValidator()
- result = validator._validateContent(
- 'test.dtd', '<!ENTITY a.translatable.string "A string">\n')
- self.assertTrue(result)
-
- def test_validate_dtd_bad(self):
- validator = self._makeValidator()
- result = validator._validateContent(
- 'test.dtd', '<!ENTIT etc.')
- self.assertFalse(result)
-
- def test_validate_xpi_manifest_good(self):
- validator = self._makeValidator()
- result = validator._validateContent(
- 'chrome.manifest', 'locale foo nl jar:chrome/nl.jar!/foo/')
- self.assertTrue(result)
-
- def test_validate_xpi_manifest_bad(self):
- # XPI manifests must not begin with newline.
- validator = self._makeValidator()
- result = validator._validateContent('chrome.manifest', '\nlocale')
- self.assertFalse(result)
-
def test_validate_po_good(self):
validator = self._makeValidator()
result = validator._validateContent('nl.po', self._strip(r"""
@@ -110,17 +83,6 @@ class TestValidateTranslationsFile(TestCase):
result = validator._validateContent('test.pot', 'garble')
self.assertFalse(result)
- def test_validate_xpi_good(self):
- validator = self._makeValidator()
- xpi_content = get_en_US_xpi_file_to_import('en-US').read()
- result = validator._validateContent('pl.xpi', xpi_content)
- self.assertTrue(result)
-
- def test_validate_xpi_bad(self):
- validator = self._makeValidator()
- result = validator._validateContent('de.xpi', 'garble')
- self.assertFalse(result)
-
def test_script(self):
test_input = os.path.join(self._findTestData(), 'minimal.pot')
script = 'scripts/rosetta/validate-translations-file.py'
diff --git a/lib/lp/translations/scripts/validate_translations_file.py b/lib/lp/translations/scripts/validate_translations_file.py
index bcf2ff6..f9d9303 100644
--- a/lib/lp/translations/scripts/validate_translations_file.py
+++ b/lib/lp/translations/scripts/validate_translations_file.py
@@ -8,18 +8,12 @@ __all__ = [
'ValidateTranslationsFile',
]
-from cStringIO import StringIO
import logging
from optparse import OptionParser
import os.path
from lp.services import scripts
from lp.translations.utilities.gettext_po_parser import POParser
-from lp.translations.utilities.mozilla_dtd_parser import DtdFile
-from lp.translations.utilities.mozilla_xpi_importer import (
- MozillaZipImportParser,
- )
-from lp.translations.utilities.xpi_manifest import XpiManifest
class UnknownFileType(Exception):
@@ -31,37 +25,19 @@ def validate_unknown_file_type(filename, content):
raise UnknownFileType("Unrecognized file type for '%s'." % filename)
-def validate_dtd(filename, content):
- """Validate XPI DTD file."""
- DtdFile(filename, filename, content)
-
-
def validate_po(filename, content):
"""Validate a gettext PO or POT file."""
POParser().parse(content)
-def validate_xpi(filename, content):
- """Validate an XPI archive."""
- MozillaZipImportParser(filename, StringIO(content))
-
-
-def validate_xpi_manifest(filename, content):
- """Validate XPI manifest."""
- XpiManifest(content)
-
-
class ValidateTranslationsFile:
"""Parse translations files to see if they are well-formed."""
name = 'validate-translations-file'
validators = {
- 'dtd': validate_dtd,
- 'manifest': validate_xpi_manifest,
'po': validate_po,
'pot': validate_po,
- 'xpi': validate_xpi,
}
def __init__(self, test_args=None):
diff --git a/lib/lp/translations/stories/standalone/xx-translations-xpi-import.txt b/lib/lp/translations/stories/standalone/xx-translations-xpi-import.txt
deleted file mode 100644
index 2f0472b..0000000
--- a/lib/lp/translations/stories/standalone/xx-translations-xpi-import.txt
+++ /dev/null
@@ -1,55 +0,0 @@
-= Demonstrate import of Firefox XPI file =
-
-To import translations into Firefox product, we must first import en-US.xpi
-file, which is an equivalent of a PO Template.
-
-Lets start with Firefox product inside trunk revision.
-
- >>> browser = setupBrowser(auth='Basic carlos@xxxxxxxxxxxxx:test')
- >>> browser.open('http://translations.launchpad.test/firefox/trunk')
-
-Since we still don't have any POTemplates assigned, we must use the general
-translations upload link.
-
- >>> browser.getLink('upload').click()
- >>> print(browser.url)
- http://translations.launchpad.test/firefox/trunk/+translations-upload
-
-Get the XPI file we are going to upload.
-
- >>> from lp.translations.utilities.tests import test_xpi_import
- >>> xpifile = test_xpi_import.get_en_US_xpi_file_to_import('en-US')
-
-Now, lets upload this file.
-
- >>> browser.getControl('File:').add_file(
- ... xpifile, 'application/zip', 'en-US.xpi')
- >>> browser.getControl('Upload').click()
-
- >>> print(browser.url)
- http://translations.launchpad.test/firefox/trunk/+translations-upload
- >>> for tag in find_tags_by_class(browser.contents, 'message'):
- ... print(extract_text(tag.renderContents()))
- Thank you for your upload. It will be automatically reviewed...
-
-Lets check the import queue to edit this entry and set the name.
-
- >>> browser.getLink('Translation Import Queue').click()
- >>> print(browser.getLink(url='en-US.xpi').url)
- http://.../en-US.xpi
- >>> browser.getLink(url='/+imports/').click()
- >>> print(browser.url)
- http://translations.launchpad.test/+imports/...
- >>> qid = int(browser.url.rsplit('/', 1)[-1])
-
-All new entries need to get a template name to identify them in the context
-where will be imported. In this case, it's 'firefox'.
-
- >>> browser.getControl('File Type').value = ['POT']
- >>> browser.getControl('Name').value = 'firefox'
- >>> browser.getControl('Translation domain').value = 'firefox'
- >>> browser.getControl('Approve').click()
- >>> print(browser.url)
- http://translations.launchpad.test/firefox/trunk/+imports
- >>> browser.getControl(name='field.status_%d' % qid).value
- ['APPROVED']
diff --git a/lib/lp/translations/tests/test_autoapproval.py b/lib/lp/translations/tests/test_autoapproval.py
index a0ec5e4..03c60b3 100644
--- a/lib/lp/translations/tests/test_autoapproval.py
+++ b/lib/lp/translations/tests/test_autoapproval.py
@@ -1179,10 +1179,7 @@ class TestAutoBlocking(TestCaseWithFactory):
translation target as `same_target_as`. This lets you create an
entry for the same translation target as another one.
"""
- if suffix == '.xpi':
- basename = 'en-US'
- else:
- basename = self.factory.getUniqueString()
+ basename = self.factory.getUniqueString()
filename = basename + suffix
if directory is None:
@@ -1227,26 +1224,6 @@ class TestAutoBlocking(TestCaseWithFactory):
self.assertEqual(len(old_blocklist), len(new_blocklist))
- def test_getBlockableDirectories_checks_xpi_templates(self):
- old_blocklist = self.queue._getBlockableDirectories()
-
- self._makeTemplateEntry(
- suffix='.xpi', status=RosettaImportStatus.BLOCKED)
-
- new_blocklist = self.queue._getBlockableDirectories()
-
- self.assertEqual(len(old_blocklist) + 1, len(new_blocklist))
-
- def test_getBlockableDirectories_ignores_xpi_translations(self):
- old_blocklist = self.queue._getBlockableDirectories()
-
- self._makeTranslationEntry(
- 'lt.xpi', status=RosettaImportStatus.BLOCKED)
-
- new_blocklist = self.queue._getBlockableDirectories()
-
- self.assertEqual(len(old_blocklist), len(new_blocklist))
-
def test_isBlockable_none(self):
blocklist = self.queue._getBlockableDirectories()
entry = self._makeTranslationEntry('nl.po')
diff --git a/lib/lp/translations/tests/test_potmsgset.py b/lib/lp/translations/tests/test_potmsgset.py
index c5aaae5..6f0a71b 100644
--- a/lib/lp/translations/tests/test_potmsgset.py
+++ b/lib/lp/translations/tests/test_potmsgset.py
@@ -15,7 +15,6 @@ from zope.security.proxy import removeSecurityProxy
from lp.app.enums import ServiceUsage
from lp.app.interfaces.launchpad import ILaunchpadCelebrities
-from lp.services.propertycache import get_property_cache
from lp.testing import TestCaseWithFactory
from lp.testing.layers import (
DatabaseFunctionalLayer,
@@ -82,55 +81,17 @@ class TestTranslationSharedPOTMsgSets(TestCaseWithFactory):
self.assertEqual(devel_potmsgsets, [self.potmsgset])
self.assertEqual(devel_potmsgsets, stable_potmsgsets)
- def test_POTMsgSetInIncompatiblePOTemplates(self):
- # Make sure a POTMsgSet cannot be used in two POTemplates with
- # different incompatible source_file_format (like XPI and PO).
- self.devel_potemplate.source_file_format = TranslationFileFormat.PO
- self.stable_potemplate.source_file_format = TranslationFileFormat.XPI
-
- potmsgset = self.potmsgset
-
- self.assertRaises(POTMsgSetInIncompatibleTemplatesError,
- potmsgset.setSequence, self.stable_potemplate, 1)
-
- # If the two file formats are compatible, it works.
- self.stable_potemplate.source_file_format = (
- TranslationFileFormat.KDEPO)
- potmsgset.setSequence(self.stable_potemplate, 1)
-
- devel_potmsgsets = list(self.devel_potemplate.getPOTMsgSets())
- stable_potmsgsets = list(self.stable_potemplate.getPOTMsgSets())
- self.assertEqual(devel_potmsgsets, stable_potmsgsets)
-
- # We hack the POTemplate manually to make data inconsistent
- # in database.
- self.stable_potemplate.source_file_format = TranslationFileFormat.XPI
- transaction.commit()
-
- # We remove the security proxy to be able to get a callable for
- # properties like `uses_english_msgids` and `singular_text`.
- naked_potmsgset = removeSecurityProxy(potmsgset)
-
- self.assertRaises(POTMsgSetInIncompatibleTemplatesError,
- naked_potmsgset.__getattribute__,
- "uses_english_msgids")
-
- self.assertRaises(POTMsgSetInIncompatibleTemplatesError,
- naked_potmsgset.__getattribute__, "singular_text")
-
def test_POTMsgSetUsesEnglishMsgids(self):
"""Test that `uses_english_msgids` property works correctly."""
+ # XXX cjwatson 2020-03-01: We currently have no file formats with
+ # importers for which uses_english_msgids would be False (XPI used
+ # to be such a case).
# Gettext PO format uses English strings as msgids.
self.devel_potemplate.source_file_format = TranslationFileFormat.PO
transaction.commit()
self.assertTrue(self.potmsgset.uses_english_msgids)
- # Mozilla XPI format doesn't use English strings as msgids.
- self.devel_potemplate.source_file_format = TranslationFileFormat.XPI
- transaction.commit()
- self.assertFalse(self.potmsgset.uses_english_msgids)
-
def test_getCurrentTranslationMessageOrDummy_returns_upstream_tm(self):
pofile = self.factory.makePOFile('nl')
message = self.factory.makeCurrentTranslationMessage(pofile=pofile)
@@ -1004,110 +965,6 @@ class TestPOTMsgSetText(TestCaseWithFactory):
english_msgid, TranslationFileFormat.PO)
self.assertEqual(english_msgid, potmsgset.singular_text)
- def test_singular_text_xpi(self):
- # Mozilla XPI format uses English strings as msgids if no English
- # pofile exists.
- symbolic_msgid = self.factory.getUniqueString()
- potmsgset = self._makePOTMsgSet(
- symbolic_msgid, TranslationFileFormat.XPI)
- self.assertEqual(symbolic_msgid, potmsgset.singular_text)
-
- def test_singular_text_xpi_english(self):
- # Mozilla XPI format uses English strings as msgids if no English
- # pofile exists.
- # POTMsgSet singular_text is read from a shared English translation.
- symbolic_msgid = self.factory.getUniqueString()
- english_msgid = self.factory.getUniqueString()
- potmsgset, potemplate = self._makePOTMsgSetAndTemplate(
- symbolic_msgid, TranslationFileFormat.XPI)
- en_pofile = self.factory.makePOFile('en', potemplate)
- self.factory.makeCurrentTranslationMessage(
- pofile=en_pofile, potmsgset=potmsgset,
- translations=[english_msgid])
-
- self.assertEqual(english_msgid, potmsgset.singular_text)
-
- def test_singular_text_xpi_english_diverged(self):
- # A diverged (translation.potemplate != None) English translation
- # is not used as a singular_text.
- symbolic_msgid = self.factory.getUniqueString()
- english_msgid = self.factory.getUniqueString()
- diverged_msgid = self.factory.getUniqueString()
- potmsgset, potemplate = self._makePOTMsgSetAndTemplate(
- symbolic_msgid, TranslationFileFormat.XPI)
- en_pofile = self.factory.makePOFile('en', potemplate)
- self.factory.makeCurrentTranslationMessage(
- pofile=en_pofile, potmsgset=potmsgset,
- translations=[english_msgid])
- self.factory.makeCurrentTranslationMessage(
- pofile=en_pofile, potmsgset=potmsgset,
- translations=[diverged_msgid], diverged=True)
-
- self.assertEqual(english_msgid, potmsgset.singular_text)
-
- def _setUpSharingWithUbuntu(self):
- """Create a potmsgset shared in upstream and Ubuntu."""
- productseries = self.factory.makeProductSeries()
-
- # Create the source package that this product is linked to.
- distroseries = self.factory.makeUbuntuDistroSeries()
- distroseries.distribution.translation_focus = distroseries
- sourcepackagename = self.factory.makeSourcePackageName()
- sourcepackage = self.factory.makeSourcePackage(
- distroseries=distroseries, sourcepackagename=sourcepackagename)
- sourcepackage.setPackaging(productseries, self.factory.makePerson())
-
- # Create two sharing templates.
- self.potmsgset, upstream_potemplate = self._makePOTMsgSetAndTemplate(
- None, TranslationFileFormat.XPI, productseries)
- ubuntu_potemplate = self.factory.makePOTemplate(
- distroseries=distroseries, sourcepackagename=sourcepackagename,
- name=upstream_potemplate.name)
- ubuntu_potemplate.source_file_format = TranslationFileFormat.XPI
- self.potmsgset.setSequence(ubuntu_potemplate, 1)
-
- # The pofile is automatically created for all sharing templates.
- self.upstream_pofile = self.factory.makePOFile(
- 'en', upstream_potemplate, create_sharing=True)
- self.ubuntu_pofile = ubuntu_potemplate.getPOFileByLang('en')
- self.assertIsNot(None, self.ubuntu_pofile)
-
- def test_singular_text_xpi_english_uses_upstream(self):
- # POTMsgSet singular_text is read from the upstream English
- # translation.
- self._setUpSharingWithUbuntu()
- # Create different "English translations" for this potmsgset.
- ubuntu_msgid = self.factory.getUniqueString()
- upstream_msgid = self.factory.getUniqueString()
-
- self.factory.makeCurrentTranslationMessage(
- pofile=self.upstream_pofile, potmsgset=self.potmsgset,
- translations=[upstream_msgid])
- self.factory.makeCurrentTranslationMessage(
- pofile=self.ubuntu_pofile, potmsgset=self.potmsgset,
- translations=[ubuntu_msgid])
-
- # makeCurrentTranslationMessage calls singular_text and caches the
- # upstream msgid, causing the test to pass even without the
- # Ubuntu message being present.
- del get_property_cache(self.potmsgset).singular_text
- self.assertEqual(upstream_msgid, self.potmsgset.singular_text)
-
- def test_singular_text_xpi_english_falls_back_to_ubuntu(self):
- # POTMsgSet singular_text is read from the Ubuntu English
- # translation if no upstream one exists. This is a safeguard against
- # old or broken data.
- self._setUpSharingWithUbuntu()
-
- # Create different "English translations" for this potmsgset.
- ubuntu_msgid = self.factory.getUniqueString()
-
- self.factory.makeCurrentTranslationMessage(
- pofile=self.ubuntu_pofile, potmsgset=self.potmsgset,
- translations=[ubuntu_msgid])
-
- self.assertEqual(ubuntu_msgid, self.potmsgset.singular_text)
-
class TestPOTMsgSetTranslationCredits(TestCaseWithFactory):
"""Test methods related to TranslationCredits."""
diff --git a/lib/lp/translations/tests/test_translationbranchapprover.py b/lib/lp/translations/tests/test_translationbranchapprover.py
index 4f309c0..43e4b36 100644
--- a/lib/lp/translations/tests/test_translationbranchapprover.py
+++ b/lib/lp/translations/tests/test_translationbranchapprover.py
@@ -111,17 +111,6 @@ class TestTranslationBranchApprover(TestCaseWithFactory):
self.assertEqual(
translation_domain, entry.potemplate.translation_domain)
- def test_new_template_domain_with_xpi(self):
- # For xpi files, template files are always called "en-US.xpi" so
- # the approver won't use that string for a domain. It'll fall
- # back to the next possibility, which is the directory.
- translation_domain = self.factory.getUniqueString()
- template_path = translation_domain + '/en-US.xpi'
- entry = self._upload_file(template_path)
- self._createApprover(template_path).approve(entry)
- self.assertEqual(
- translation_domain, entry.potemplate.translation_domain)
-
def test_template_name(self):
# The name is derived from the file name and must be a valid name.
translation_domain = (u'Invalid-Name_with illegal#Characters')
diff --git a/lib/lp/translations/tests/test_translationimportqueue.py b/lib/lp/translations/tests/test_translationimportqueue.py
index e3b3e4c..c4ed52e 100644
--- a/lib/lp/translations/tests/test_translationimportqueue.py
+++ b/lib/lp/translations/tests/test_translationimportqueue.py
@@ -407,7 +407,6 @@ class TestTranslationImportQueue(TestCaseWithFactory):
files = dict((
self._makeFile('pot'),
self._makeFile('po'),
- self._makeFile('xpi'),
))
tarfile_content = LaunchpadWriteTarFile.files_to_stream(files)
self.import_queue.addOrUpdateEntriesFromTarball(
diff --git a/lib/lp/translations/utilities/mozilla_dtd_parser.py b/lib/lp/translations/utilities/mozilla_dtd_parser.py
deleted file mode 100644
index bdc4782..0000000
--- a/lib/lp/translations/utilities/mozilla_dtd_parser.py
+++ /dev/null
@@ -1,150 +0,0 @@
-# Copyright 2010 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Importer for DTD files as found in XPI archives."""
-
-__metaclass__ = type
-__all__ = [
- 'DtdFile'
- ]
-
-from old_xmlplus.parsers.xmlproc import (
- dtdparser,
- utils,
- xmldtd,
- )
-
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatInvalidInputError,
- TranslationFormatSyntaxError,
- )
-from lp.translations.interfaces.translations import TranslationConstants
-from lp.translations.utilities.translation_common_format import (
- TranslationMessageData,
- )
-
-
-class MozillaDtdConsumer(xmldtd.WFCDTD):
- """Mozilla DTD translatable message parser.
-
- msgids are stored as entities. This class extracts it along
- with translations, comments and source references.
- """
- def __init__(self, parser, filename, chrome_path, messages):
- self.started = False
- self.last_comment = None
- self.chrome_path = chrome_path
- self.messages = messages
- self.filename = filename
- xmldtd.WFCDTD.__init__(self, parser)
-
- def dtd_start(self):
- """See `xmldtd.WFCDTD`."""
- self.started = True
-
- def dtd_end(self):
- """See `xmldtd.WFCDTD`."""
- self.started = False
-
- def handle_comment(self, contents):
- """See `xmldtd.WFCDTD`."""
- if not self.started:
- return
-
- if self.last_comment is not None:
- self.last_comment += contents
- elif len(contents) > 0:
- self.last_comment = contents
-
- if self.last_comment and not self.last_comment.endswith('\n'):
- # Comments must end always with a new line.
- self.last_comment += '\n'
-
- def new_general_entity(self, name, value):
- """See `xmldtd.WFCDTD`."""
- if not self.started:
- return
-
- message = TranslationMessageData()
- message.msgid_singular = name
- # CarlosPerelloMarin 20070326: xmldtd parser does an inline
- # parsing which means that the content is all in a single line so we
- # don't have a way to show the line number with the source reference.
- message.file_references_list = ["%s(%s)" % (self.filename, name)]
- message.addTranslation(TranslationConstants.SINGULAR_FORM, value)
- message.singular_text = value
- message.context = self.chrome_path
- message.source_comment = self.last_comment
- self.messages.append(message)
- self.started += 1
- self.last_comment = None
-
-
-class DtdErrorHandler(utils.ErrorCounter):
- """Error handler for the DTD parser."""
- filename = None
-
- def error(self, msg):
- raise TranslationFormatSyntaxError(
- filename=self.filename, message=msg)
-
- def fatal(self, msg):
- raise TranslationFormatInvalidInputError(
- filename=self.filename, message=msg)
-
-
-class DummyDtdFile:
- """"File" returned when DTD SYSTEM entity tries to include a file."""
- done = False
-
- def read(self, *args, **kwargs):
- """Minimally satisfy attempt to read an included DTD file."""
- if self.done:
- return ''
- else:
- self.done = True
- return '<!-- SYSTEM entities not supported. -->'
-
- def close(self):
- """Satisfy attempt to close file."""
- pass
-
-
-class DtdInputSourceFactoryStub:
- """Replace the class the DTD parser uses to include other DTD files."""
-
- def create_input_source(self, sysid):
- """Minimally satisfy attempt to open an included DTD file.
-
- This is called when the DTD parser hits a SYSTEM entity.
- """
- return DummyDtdFile()
-
-
-class DtdFile:
- """Class for reading translatable messages from a .dtd file.
-
- It uses DTDParser which fills self.messages with parsed messages.
- """
- def __init__(self, filename, chrome_path, content):
- self.messages = []
- self.filename = filename
- self.chrome_path = chrome_path
-
- # .dtd files are supposed to be using UTF-8 encoding, if the file is
- # using another encoding, it's against the standard so we reject it
- try:
- content = content.decode('utf-8')
- except UnicodeDecodeError:
- raise TranslationFormatInvalidInputError(
- 'Content is not valid UTF-8 text')
-
- error_handler = DtdErrorHandler()
- error_handler.filename = filename
-
- parser = dtdparser.DTDParser()
- parser.set_error_handler(error_handler)
- parser.set_inputsource_factory(DtdInputSourceFactoryStub())
- dtd = MozillaDtdConsumer(parser, filename, chrome_path, self.messages)
- parser.set_dtd_consumer(dtd)
- parser.parse_string(content)
diff --git a/lib/lp/translations/utilities/mozilla_xpi_importer.py b/lib/lp/translations/utilities/mozilla_xpi_importer.py
deleted file mode 100644
index 3c76c79..0000000
--- a/lib/lp/translations/utilities/mozilla_xpi_importer.py
+++ /dev/null
@@ -1,423 +0,0 @@
-# Copyright 2009-2010 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-__metaclass__ = type
-
-__all__ = [
- 'MozillaXpiImporter',
- 'MozillaZipImportParser',
- ]
-
-from cStringIO import StringIO
-import textwrap
-
-from zope.component import getUtility
-from zope.interface import implementer
-
-from lp.services.librarian.interfaces.client import ILibrarianClient
-from lp.translations.interfaces.translationfileformat import (
- TranslationFileFormat,
- )
-from lp.translations.interfaces.translationimporter import (
- ITranslationFormatImporter,
- TranslationFormatInvalidInputError,
- TranslationFormatSyntaxError,
- )
-from lp.translations.interfaces.translations import TranslationConstants
-from lp.translations.utilities.mozilla_dtd_parser import DtdFile
-from lp.translations.utilities.mozilla_zip import MozillaZipTraversal
-from lp.translations.utilities.translation_common_format import (
- TranslationFileData,
- TranslationMessageData,
- )
-from lp.translations.utilities.xpi_header import XpiHeader
-
-
-def add_source_comment(message, comment):
- """Add the given comment inside message.source_comment."""
- if message.source_comment:
- message.source_comment += comment
- else:
- message.source_comment = comment
-
- if not message.source_comment.endswith('\n'):
- message.source_comment += '\n'
-
-
-class MozillaZipImportParser(MozillaZipTraversal):
- """XPI and jar parser for import purposes.
-
- Looks for DTD and properties files, and parses them for messages.
- All messages found are left in `self.messages`.
- """
-
- # List of ITranslationMessageData representing messages found.
- messages = None
-
- def _begin(self):
- """Overridable hook for `MozillaZipTraversal`."""
- self.messages = []
-
- def _finish(self):
- """Overridable hook for `MozillaZipTraversal`."""
- # Eliminate duplicate messages.
- seen_messages = set()
- deletions = []
- for index, message in enumerate(self.messages):
- identifier = (message.msgid_singular, message.context)
- if identifier in seen_messages:
- # This message is a duplicate. Mark it for removal.
- deletions.append(index)
- else:
- seen_messages.add(identifier)
- for index in reversed(deletions):
- del self.messages[index]
-
- for message in self.messages:
- message.file_references = ', '.join(message.file_references_list)
-
- def _processTranslatableFile(self, entry, locale_code, xpi_path,
- chrome_path, filename_suffix):
- """Overridable hook for `MozillaZipTraversal`.
-
- This implementation is only interested in DTD and properties
- files.
- """
- if filename_suffix == '.dtd':
- parser = DtdFile
- elif filename_suffix == '.properties':
- parser = PropertyFile
- else:
- # We're not interested in other file types here.
- return
-
- # Parse file, subsume its messages.
- content = self.archive.read(entry)
- parsed_file = parser(
- filename=xpi_path, chrome_path=chrome_path, content=content)
- if parsed_file is not None:
- self.extend(parsed_file.messages)
-
- def _isTemplate(self):
- """Is this a template?"""
- name = self.filename
- return name is not None and name.startswith('en-US.xpi')
-
- def _processNestedJar(self, zip_instance):
- """Overridable hook for `MozillaZipTraversal`.
-
- This implementation complements `self.messages` with those found in
- the jar file we just parsed.
- """
- self.extend(zip_instance.messages)
-
- def _isCommandKeyMessage(self, message):
- """Whether the message represents a command key shortcut."""
- return (
- self._isTemplate() and
- message.translations and (
- message.msgid_singular.endswith('.commandkey') or
- message.msgid_singular.endswith('.key')))
-
- def _isAccessKeyMessage(self, message):
- """Whether the message represents an access key shortcut."""
- return (
- self._isTemplate() and
- message.translations and (
- message.msgid_singular.endswith('.accesskey')))
-
- def extend(self, newdata):
- """Complement `self.messages` with messages found in contained file.
-
- :param newdata: a sequence representing the messages found in a
- contained file.
- """
- for message in newdata:
- # Special case accesskeys and commandkeys:
- # these are single letter messages, lets display
- # the value as a source comment.
- if self._isCommandKeyMessage(message):
- comment = u'\n'.join(textwrap.wrap(
- u"""Select the shortcut key that you want to use. It
- should be translated, but often shortcut keys (for
- example Ctrl + KEY) are not changed from the original. If
- a translation already exists, please don't change it if
- you are not sure about it. Please find the context of
- the key from the end of the 'Located in' text below."""))
- add_source_comment(message, comment)
- elif self._isAccessKeyMessage(message):
- comment = u'\n'.join(textwrap.wrap(
- u"""Select the access key that you want to use. These have
- to be translated in a way that the selected character is
- present in the translated string of the label being
- referred to, for example 'i' in 'Edit' menu item in
- English. If a translation already exists, please don't
- change it if you are not sure about it. Please find the
- context of the key from the end of the 'Located in' text
- below."""))
- add_source_comment(message, comment)
- self.messages.append(message)
-
-
-def valid_property_msgid(msgid):
- """Whether the given msgid follows the restrictions to be valid.
-
- Checks done are:
- - It cannot have white spaces.
- """
- return u' ' not in msgid
-
-
-class PropertyFile:
- """Class for reading translatable messages from a .properties file.
-
- The file format is described at:
- http://www.mozilla.org/projects/l10n/mlp_chrome.html#text
- """
-
- license_block_text = u'END LICENSE BLOCK'
-
- def __init__(self, filename, chrome_path, content):
- """Constructs a dictionary from a .properties file.
-
- :arg filename: The file name where the content came from.
- :arg content: The file content that we want to parse.
- """
- self.filename = filename
- self.chrome_path = chrome_path
- self.messages = []
-
- # Parse the content.
- self.parse(content)
-
- def parse(self, content):
- """Parse given content as a property file.
-
- Once the parse is done, self.messages has a list of the available
- `ITranslationMessageData`s.
- """
-
- # .properties files are supposed to be unicode-escaped, but we know
- # that there are some .xpi language packs that instead, use UTF-8.
- # That's against the specification, but Mozilla applications accept
- # it anyway, so we try to support it too.
- # To do this support, we read the text as being in UTF-8
- # because unicode-escaped looks like ASCII files.
- try:
- content = content.decode('utf-8')
- except UnicodeDecodeError:
- raise TranslationFormatInvalidInputError(
- 'Content is not valid unicode-escaped text')
-
- line_num = 0
- is_multi_line_comment = False
- last_comment = None
- last_comment_line_num = 0
- ignore_comment = False
- is_message = False
- translation = u''
- for line in content.splitlines():
- # Now, to "normalize" all to the same encoding, we encode to
- # unicode-escape first, and then decode it to unicode
- # XXX: Danilo 2006-08-01: we _might_ get performance
- # improvements if we reimplement this to work directly,
- # though, it will be hard to beat C-based de/encoder.
- # This call unescapes everything so we don't need to care about
- # quotes escaping.
- try:
- string = line.encode('raw-unicode_escape')
- line = string.decode('unicode_escape')
- except UnicodeDecodeError as exception:
- raise TranslationFormatInvalidInputError(
- filename=self.filename, line_number=line_num,
- message=str(exception))
-
- line_num += 1
- if not is_multi_line_comment:
- # Remove any white space before the useful data, like
- # ' # foo'.
- line = line.lstrip()
- if len(line) == 0:
- # It's an empty line. Reset any previous comment we have.
- last_comment = None
- last_comment_line_num = 0
- ignore_comment = False
- elif line.startswith(u'#') or line.startswith(u'//'):
- # It's a whole line comment.
- ignore_comment = False
- line = line[1:].strip()
- if last_comment:
- last_comment += line
- elif len(line) > 0:
- last_comment = line
-
- if last_comment and not last_comment.endswith('\n'):
- # Comments must end always with a new line.
- last_comment += '\n'
-
- last_comment_line_num = line_num
- continue
-
- # Unescaped URLs are a common mistake: the "//" starts an
- # end-of-line comment. To work around that, treat "://" as
- # a special case.
- just_saw_colon = False
-
- while line:
- if is_multi_line_comment:
- if line.startswith(u'*/'):
- # The comment ended, we jump the closing tag and
- # continue with the parsing.
- line = line[2:]
- is_multi_line_comment = False
- last_comment_line_num = line_num
- if ignore_comment:
- last_comment = None
- ignore_comment = False
-
- # Comments must end always with a new line.
- last_comment += '\n'
- elif line.startswith(self.license_block_text):
- # It's a comment with a licence notice, this
- # comment can be ignored.
- ignore_comment = True
- # Jump the whole tag
- line = line[len(self.license_block_text):]
- else:
- # Store the character.
- if last_comment is None:
- last_comment = line[0]
- elif last_comment_line_num == line_num:
- last_comment += line[0]
- else:
- last_comment = u'%s\n%s' % (last_comment, line[0])
- last_comment_line_num = line_num
- # Jump the processed char.
- line = line[1:]
- continue
- elif line.startswith(u'/*'):
- # It's a multi line comment
- is_multi_line_comment = True
- ignore_comment = False
- last_comment_line_num = line_num
- # Jump the comment starting tag
- line = line[2:]
- continue
- elif line.startswith(u'//') and not just_saw_colon:
- # End-of-line comment.
- last_comment = '%s\n' % line[2:].strip()
- last_comment_line_num = line_num
- # On to next line.
- break
- elif is_message:
- # Store the char and continue.
- head_char = line[0]
- translation += head_char
- line = line[1:]
- just_saw_colon = (head_char == ':')
- continue
- elif u'=' in line:
- # Looks like a message string.
- (key, value) = line.split('=', 1)
- # Remove leading and trailing white spaces.
- key = key.strip()
-
- if valid_property_msgid(key):
- is_message = True
- # Jump the msgid, control chars and leading white
- # space.
- line = value.lstrip()
- continue
- else:
- raise TranslationFormatSyntaxError(
- line_number=line_num,
- message=u"invalid msgid: '%s'" % key)
- else:
- # Got a line that is not a valid message nor a valid
- # comment. Ignore it because main en-US.xpi catalog from
- # Firefox has such line/error. We follow the 'be strict
- # with what you export, be permisive with what you import'
- # policy.
- break
- if is_message:
- # We just parsed a message, so we need to add it to the list
- # of messages.
- if ignore_comment or last_comment_line_num < line_num - 1:
- # We must ignore the comment or either the comment is not
- # the last thing before this message or is not in the same
- # line as this message.
- last_comment = None
- ignore_comment = False
-
- message = TranslationMessageData()
- message.msgid_singular = key
- message.context = self.chrome_path
- message.file_references_list = [
- "%s:%d(%s)" % (self.filename, line_num, key)]
- value = translation.strip()
- message.addTranslation(
- TranslationConstants.SINGULAR_FORM, value)
- message.singular_text = value
- message.source_comment = last_comment
- self.messages.append(message)
-
- # Reset status vars.
- last_comment = None
- last_comment_line_num = 0
- is_message = False
- translation = u''
-
-
-@implementer(ITranslationFormatImporter)
-class MozillaXpiImporter:
- """Support class to import Mozilla .xpi files."""
-
- def __init__(self):
- self.basepath = None
- self.productseries = None
- self.distroseries = None
- self.sourcepackagename = None
- self.by_maintainer = False
- self._translation_file = None
-
- def getFormat(self, file_contents):
- """See `ITranslationFormatImporter`."""
- return TranslationFileFormat.XPI
-
- priority = 0
-
- # using "application/x-xpinstall" would trigger installation in
- # firefox.
- content_type = 'application/zip'
-
- file_extensions = ['.xpi']
- template_suffix = 'en-US.xpi'
-
- uses_source_string_msgids = True
-
- def parse(self, translation_import_queue_entry):
- """See `ITranslationFormatImporter`."""
- self._translation_file = TranslationFileData()
- self.basepath = translation_import_queue_entry.path
- self.productseries = translation_import_queue_entry.productseries
- self.distroseries = translation_import_queue_entry.distroseries
- self.sourcepackagename = (
- translation_import_queue_entry.sourcepackagename)
- self.by_maintainer = translation_import_queue_entry.by_maintainer
-
- librarian_client = getUtility(ILibrarianClient)
- content = librarian_client.getFileByAlias(
- translation_import_queue_entry.content.id).read()
-
- parser = MozillaZipImportParser(self.basepath, StringIO(content))
- if parser.header is None:
- raise TranslationFormatInvalidInputError("No install.rdf found")
-
- self._translation_file.header = parser.header
- self._translation_file.messages = parser.messages
-
- return self._translation_file
-
- def getHeaderFromString(self, header_string):
- """See `ITranslationFormatImporter`."""
- return XpiHeader(header_string)
diff --git a/lib/lp/translations/utilities/mozilla_zip.py b/lib/lp/translations/utilities/mozilla_zip.py
deleted file mode 100644
index 489a6a2..0000000
--- a/lib/lp/translations/utilities/mozilla_zip.py
+++ /dev/null
@@ -1,170 +0,0 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-__metaclass__ = type
-
-__all__ = [
- 'MozillaZipTraversal',
- ]
-
-from cStringIO import StringIO
-from os.path import (
- basename,
- splitext,
- )
-from zipfile import (
- BadZipfile,
- ZipFile,
- )
-
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatInvalidInputError,
- )
-from lp.translations.utilities.xpi_header import XpiHeader
-from lp.translations.utilities.xpi_manifest import (
- make_jarpath,
- XpiManifest,
- )
-
-
-class MozillaZipTraversal:
- """Traversal of an XPI file, or a jar file inside an XPI file.
-
- To traverse and process an XPI file, derive a class from this one
- and replace any hooks that you may need.
-
- If an XPI manifest is provided, traversal will be restricted to
- directories it lists as containing localizable resources.
- """
-
- def __init__(self, filename, archive, xpi_path=None, manifest=None):
- """Open zip (or XPI, or jar) file and scan its contents.
-
- :param filename: Name of this zip (XPI/jar) archive.
- :param archive: File-like object containing this zip archive.
- :param xpi_path: Full path of this file inside the XPI archive.
- Leave out for the XPI archive itself.
- :param manifest: `XpiManifest` representing the XPI archive's
- manifest file, if any.
- """
- self.filename = filename
- self.header = None
- self.last_translator = None
- self.manifest = manifest
- try:
- self.archive = ZipFile(archive, 'r')
- except BadZipfile as exception:
- raise TranslationFormatInvalidInputError(
- filename=filename, message=str(exception))
-
- if xpi_path is None:
- # This is the main XPI file.
- xpi_path = ''
- contained_files = set(self.archive.namelist())
- if manifest is None:
- # Look for a manifest.
- for filename in ['chrome.manifest', 'en-US.manifest']:
- if filename in contained_files:
- manifest_content = self.archive.read(filename)
- self.manifest = XpiManifest(manifest_content)
- break
- if 'install.rdf' in contained_files:
- rdf_content = self.archive.read('install.rdf')
- self.header = XpiHeader(rdf_content)
-
- # Strip trailing newline to avoid doubling it.
- xpi_path = xpi_path.rstrip('/')
-
- self._begin()
-
- # Process zipped files. Sort by path to keep ordering deterministic.
- # Ordering matters in sequence numbering (which in turn shows up in
- # the UI), but also for consistency in duplicates resolution and for
- # automated testing.
- for entry in sorted(self.archive.namelist()):
- self._processEntry(entry, xpi_path)
-
- self._finish()
-
- def _processEntry(self, entry, xpi_path):
- """Read one zip archive entry, figure out what to do with it."""
- rootname, suffix = splitext(entry)
- if basename(rootname) == '':
- # If filename starts with a dot, that's not really a suffix.
- suffix = ''
-
- if suffix == '.jar':
- jarpath = make_jarpath(xpi_path, entry)
- if not self.manifest or self.manifest.containsLocales(jarpath):
- # If this is a jar file that may contain localizable
- # resources, don't process it in the normal way; recurse
- # by creating another parser instance.
- content = self.archive.read(entry)
- nested_instance = self.__class__(
- filename=entry, archive=StringIO(content),
- xpi_path=jarpath, manifest=self.manifest)
-
- self._processNestedJar(nested_instance)
- return
-
- # Construct XPI path; identical to "entry" if previous xpi_path
- # was empty. XPI paths use slashes as separators, regardless of
- # what the native filesystem uses.
- xpi_path = '/'.join([xpi_path, entry]).lstrip('/')
-
- if self.manifest is None:
- # No manifest, so we don't have chrome paths. Process
- # everything just to be sure.
- chrome_path = None
- locale_code = None
- else:
- chrome_path, locale_code = self.manifest.getChromePathAndLocale(
- xpi_path)
- if chrome_path is None:
- # Not in a directory containing localizable resources.
- return
-
- self._processTranslatableFile(
- entry, locale_code, xpi_path, chrome_path, suffix)
-
- def _begin(self):
- """Overridable hook: optional pre-traversal actions."""
-
- def _processTranslatableFile(self, entry, locale_code, xpi_path,
- chrome_path, filename_suffix):
- """Overridable hook: process a file entry.
-
- Called only for files that may be localizable. If there is a
- manifest, that means the file must be in a location (or subtree)
- named by a "locale" entry.
-
- :param entry: Full name of file inside this zip archive,
- including path relative to the archive's root.
- :param locale_code: Code for locale this file belongs to, e.g.
- "en-US".
- :param xpi_path: Full path of this file inside the XPI archive,
- e.g. "jar:locale/en-US.jar!/data/messages.dtd".
- :param chrome_path: File's chrome path. This is a kind of
- "normalized" path used in XPI to describe a virtual
- directory hierarchy. The zip archive's actual layout (which
- the XPI paths describe) may be different.
- :param filename_suffix: File name suffix or "extension" of the
- translatable file. This may be e.g. ".dtd" or ".xhtml," or
- the empty string if the filename does not contain a dot.
- """
- raise NotImplementedError(
- "XPI traversal class provides no _processTranslatableFile().")
-
- def _processNestedJar(self, zip_instance):
- """Overridable hook: handle a nested jar file.
-
- :param zip_instance: An instance of the same class as self, which
- has just parsed the nested jar file.
- """
- raise NotImplementedError(
- "XPI traversal class provides no _processNestedJar().")
-
- def _finish(self):
- """Overridable hook: post-traversal actions."""
- raise NotImplementedError(
- "XPI traversal class provides no _finish().")
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/chrome.manifest b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/chrome.manifest
deleted file mode 100644
index 99c5dc3..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/chrome.manifest
+++ /dev/null
@@ -1,5 +0,0 @@
-locale mac en-US jar:chrome/en-US.jar!/mac/
-locale unix en-US jar:chrome/en-US.jar!/unix/
-locale win en-US jar:chrome/en-US.jar!/win/
-override chrome://foo/bar/splat.dtd
-locale main en-US jar:chrome/en-US.jar!/
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/mac/extra.dtd b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/mac/extra.dtd
deleted file mode 100644
index ae5af22..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/mac/extra.dtd
+++ /dev/null
@@ -1,2 +0,0 @@
-<!-- This message id also occurs elsewhere in this file -->
-<!ENTITY foozilla.clashing.key "This message is Mac-specific, and comes from DTD.">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/mac/extra.properties b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/mac/extra.properties
deleted file mode 100644
index cb23d48..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/mac/extra.properties
+++ /dev/null
@@ -1 +0,0 @@
-foozilla.clashing.key=This message is Mac-specific, and comes from properties.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/main.dtd b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/main.dtd
deleted file mode 100644
index 9e608fa..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/main.dtd
+++ /dev/null
@@ -1,6 +0,0 @@
-<!-- This message id also occurs elsewhere in this file -->
-<!ENTITY foozilla.regular.message "A non-clashing message.">
-<!ENTITY foozilla.clashing.key "This message is in the main DTD.">
-
-<!-- Clashing msgid within same file. Should be ignored. -->
-<!ENTITY foozilla.regular.message "This message should be ignored.">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/main.properties b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/main.properties
deleted file mode 100644
index 0a317b1..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/main.properties
+++ /dev/null
@@ -1 +0,0 @@
-foozilla.clashing.key=This message is in the main properties file.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/unix/extra.dtd b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/unix/extra.dtd
deleted file mode 100644
index 7388985..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/unix/extra.dtd
+++ /dev/null
@@ -1,2 +0,0 @@
-<!-- This message id also occurs elsewhere in this file -->
-<!ENTITY foozilla.clashing.key "This message is Unix-specific, and comes from DTD.">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/unix/extra.properties b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/unix/extra.properties
deleted file mode 100644
index 6888f4d..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/unix/extra.properties
+++ /dev/null
@@ -1 +0,0 @@
-foozilla.clashing.key=This message is Unix-specific, and comes from properties.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/win/extra.dtd b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/win/extra.dtd
deleted file mode 100644
index ee2dbd6..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/win/extra.dtd
+++ /dev/null
@@ -1,2 +0,0 @@
-<!-- This message id also occurs elsewhere in this file -->
-<!ENTITY foozilla.clashing.key "This message is Windows-specific, and comes from DTD.">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/win/extra.properties b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/win/extra.properties
deleted file mode 100644
index aacf3d8..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/en-US-jar/win/extra.properties
+++ /dev/null
@@ -1 +0,0 @@
-foozilla.clashing.key=This message is Windows-specific, and comes from properties.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/install.rdf b/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/install.rdf
deleted file mode 100644
index 872390f..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/clashing_ids/install.rdf
+++ /dev/null
@@ -1,19 +0,0 @@
-<?xml version="1.0"?>
-<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
- xmlns:em="http://www.mozilla.org/2004/em-rdf#";>
- <Description about="urn:mozilla:install-manifest"
- em:id="langpack-en-US@xxxxxxxxxxxxxxxxxxx"
- em:name="English U.S. (en-US) Language Pack"
- em:version="2.0"
- em:type="8"
- em:creator="Jeroen Vermeulen">
-
- <em:targetApplication>
- <Description>
- <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id><!-- firefox -->
- <em:minVersion>2.0</em:minVersion>
- <em:maxVersion>2.0.0.*</em:maxVersion>
- </Description>
- </em:targetApplication>
- </Description>
-</RDF>
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/chrome.manifest b/lib/lp/translations/utilities/tests/firefox-data/en-US/chrome.manifest
deleted file mode 100644
index 5c362be..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/chrome.manifest
+++ /dev/null
@@ -1 +0,0 @@
-locale main en-US jar:chrome/en-US.jar!/
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/copyover3.png b/lib/lp/translations/utilities/tests/firefox-data/en-US/copyover3.png
deleted file mode 100644
index cdac869..0000000
Binary files a/lib/lp/translations/utilities/tests/firefox-data/en-US/copyover3.png and /dev/null differ
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/copyover1.foo b/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/copyover1.foo
deleted file mode 100644
index d39d61f..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/copyover1.foo
+++ /dev/null
@@ -1,3 +0,0 @@
-This file is copied directly over to resulting translated XPI files.
-
-We only need to make sure the contents doesn't change a bit.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/copyover2.foo b/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/copyover2.foo
deleted file mode 100644
index c82082f..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/copyover2.foo
+++ /dev/null
@@ -1,7 +0,0 @@
-This is another file which should be blindly copied without any
-changes in the content, no matter how much one tries.
-
-And for some binary checks, lets add some UTF-8 encoded data. For
-example, name of "Carlos Perelló Marín" would be written as "Карлос
-Перељо Марин" in Serbian (which is phonetic, and is read exactly the
-same minus accents).
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/test2.dtd b/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/test2.dtd
deleted file mode 100644
index 4efb0fe..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/test2.dtd
+++ /dev/null
@@ -1,5 +0,0 @@
-<!-- This is a DTD file inside a subdirectory -->
-
-<!ENTITY foozilla.menu.title "MENU">
-<!ENTITY foozilla.menu.accesskey "M">
-<!ENTITY foozilla.menu.commandkey "m">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/test2.properties b/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/test2.properties
deleted file mode 100644
index e6c200b..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/subdir/test2.properties
+++ /dev/null
@@ -1,6 +0,0 @@
-# This is a Properties file inside a subdirectory
-
-# Translators, what you are seeing now is a lovely,
-# awesome, multiline comment aimed at you directly
-# from the streets of a .properties file
-foozilla_something=SomeZilla
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/test1.dtd b/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/test1.dtd
deleted file mode 100644
index 0828f76..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/test1.dtd
+++ /dev/null
@@ -1,6 +0,0 @@
-<!ENTITY foozilla.name "FooZilla!">
-<!-- Translators, don't play with fire! -->
-<!ENTITY foozilla.play.fire "Do you want to play with fire?">
-<!-- This is just a comment, not a comment for translators -->
-
-<!ENTITY foozilla.play.ice "Play with ice?">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/test1.properties b/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/test1.properties
deleted file mode 100644
index a4b7b40..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/en-US-jar/test1.properties
+++ /dev/null
@@ -1,5 +0,0 @@
-foozilla.title=FooZilla Zilla Thingy
-# Translators, if you're older than six, don't translate this
-foozilla.happytitle=http://foozillingy.happy.net/
-foozilla.nocomment=No Comment // (Except this one)
-foozilla.utf8=Дан=Day
diff --git a/lib/lp/translations/utilities/tests/firefox-data/en-US/install.rdf b/lib/lp/translations/utilities/tests/firefox-data/en-US/install.rdf
deleted file mode 100644
index a34cc54..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/en-US/install.rdf
+++ /dev/null
@@ -1,21 +0,0 @@
-<?xml version="1.0"?>
-<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
- xmlns:em="http://www.mozilla.org/2004/em-rdf#";>
- <Description about="urn:mozilla:install-manifest"
- em:id="langpack-en-US@xxxxxxxxxxxxxxxxxxx"
- em:name="English U.S. (en-US) Language Pack"
- em:version="2.0"
- em:type="8"
- em:creator="Danilo Šegan">
- <em:contributor>Данило Шеган</em:contributor>
- <em:contributor>Carlos Perelló Marín <carlos@xxxxxxxxxxxxx></em:contributor>
-
- <em:targetApplication>
- <Description>
- <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id><!-- firefox -->
- <em:minVersion>2.0</em:minVersion>
- <em:maxVersion>2.0.0.*</em:maxVersion>
- </Description>
- </em:targetApplication>
- </Description>
-</RDF>
diff --git a/lib/lp/translations/utilities/tests/firefox-data/no-manifest/en-US-jar/file.txt b/lib/lp/translations/utilities/tests/firefox-data/no-manifest/en-US-jar/file.txt
deleted file mode 100644
index 2d4b8bd..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/no-manifest/en-US-jar/file.txt
+++ /dev/null
@@ -1 +0,0 @@
-This is a translatable text file.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/no-manifest/no-jar.txt b/lib/lp/translations/utilities/tests/firefox-data/no-manifest/no-jar.txt
deleted file mode 100644
index 80a1d56..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/no-manifest/no-jar.txt
+++ /dev/null
@@ -1 +0,0 @@
-This is a file in an XPI but not in a jar.
diff --git a/lib/lp/translations/utilities/tests/firefox-data/system-entity/chrome.manifest b/lib/lp/translations/utilities/tests/firefox-data/system-entity/chrome.manifest
deleted file mode 100644
index 5c362be..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/system-entity/chrome.manifest
+++ /dev/null
@@ -1 +0,0 @@
-locale main en-US jar:chrome/en-US.jar!/
diff --git a/lib/lp/translations/utilities/tests/firefox-data/system-entity/en-US-jar/test.dtd b/lib/lp/translations/utilities/tests/firefox-data/system-entity/en-US-jar/test.dtd
deleted file mode 100644
index 7af5a5a..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/system-entity/en-US-jar/test.dtd
+++ /dev/null
@@ -1,9 +0,0 @@
-<!-- Test SYSTEM handling. -->
-
-<!ENTITY firststring "First translatable string">
-
-<!ENTITY % includedFile SYSTEM "chrome://includedFile.dtd">
-%includedFile;
-
-<!-- Parser will only get here if that last tag didn't break things. -->
-<!ENTITY secondstring "Second translatable string">
diff --git a/lib/lp/translations/utilities/tests/firefox-data/system-entity/install.rdf b/lib/lp/translations/utilities/tests/firefox-data/system-entity/install.rdf
deleted file mode 100644
index 1484c26..0000000
--- a/lib/lp/translations/utilities/tests/firefox-data/system-entity/install.rdf
+++ /dev/null
@@ -1,17 +0,0 @@
-<?xml version="1.0"?>
-<RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
- xmlns:em="http://www.mozilla.org/2004/em-rdf#";>
- <Description about="urn:mozilla:install-manifest"
- em:id="langpack-en-US@xxxxxxxxxxxxxxxxxxx"
- em:name="English U.S. (en-US) Language Pack"
- em:version="2.0"
- em:type="8">
- <em:targetApplication>
- <Description>
- <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id><!-- firefox -->
- <em:minVersion>2.0</em:minVersion>
- <em:maxVersion>2.0.0.*</em:maxVersion>
- </Description>
- </em:targetApplication>
- </Description>
-</RDF>
diff --git a/lib/lp/translations/utilities/tests/test_mozilla_xpi_importer.py b/lib/lp/translations/utilities/tests/test_mozilla_xpi_importer.py
deleted file mode 100644
index ce147a7..0000000
--- a/lib/lp/translations/utilities/tests/test_mozilla_xpi_importer.py
+++ /dev/null
@@ -1,48 +0,0 @@
-# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Mozilla XPI importer tests."""
-
-__metaclass__ = type
-
-from io import BytesIO
-import unittest
-
-from zope.interface.verify import verifyObject
-
-from lp.testing.layers import LaunchpadZopelessLayer
-from lp.translations.interfaces.translationfileformat import (
- TranslationFileFormat,
- )
-from lp.translations.interfaces.translationimporter import (
- ITranslationFormatImporter,
- )
-from lp.translations.utilities.mozilla_xpi_importer import MozillaXpiImporter
-
-
-class MozillaXpiImporterTestCase(unittest.TestCase):
- """Class test for mozilla's .xpi file imports"""
-
- layer = LaunchpadZopelessLayer
-
- def setUp(self):
- self.importer = MozillaXpiImporter()
-
- def testInterface(self):
- """Check whether the object follows the interface."""
- self.assertTrue(
- verifyObject(ITranslationFormatImporter, self.importer))
-
- def testFormat(self):
- """Check that MozillaXpiImporter handles the XPI file format."""
- format = self.importer.getFormat(BytesIO(b''))
- self.assertTrue(
- format == TranslationFileFormat.XPI,
- 'MozillaXpiImporter format expected XPI but got %s' % format.name)
-
- def testHasAlternativeMsgID(self):
- """Check that MozillaXpiImporter has an alternative msgid."""
- self.assertTrue(
- self.importer.uses_source_string_msgids,
- "MozillaXpiImporter format says it's not using alternative msgid"
- " when it really does!")
diff --git a/lib/lp/translations/utilities/tests/test_mozilla_zip.py b/lib/lp/translations/utilities/tests/test_mozilla_zip.py
deleted file mode 100644
index d7eee80..0000000
--- a/lib/lp/translations/utilities/tests/test_mozilla_zip.py
+++ /dev/null
@@ -1,113 +0,0 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""`MozillaZipTraversal` tests."""
-
-__metaclass__ = type
-
-import unittest
-
-from lp.testing.layers import LaunchpadZopelessLayer
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatInvalidInputError,
- )
-from lp.translations.utilities.mozilla_zip import MozillaZipTraversal
-from lp.translations.utilities.tests.xpi_helpers import (
- get_en_US_xpi_file_to_import,
- )
-
-
-class TraversalRecorder(MozillaZipTraversal):
- """XPI "parser": records traversal of an XPI or jar file.
-
- Does nothing but keep track of the structure of nested zip files it
- traverses, and the various parameters for each translatable file.
-
- Produces a nice list of tuples (representing parameters for a
- translatable file) and lists (representing nested jar files). Each
- zip file's traversal, including nested ones, is concluded with a
- string containing a full stop (".").
- """
- traversal = None
-
- def _begin(self):
- self.traversal = []
-
- def _processTranslatableFile(self, entry, locale_code, xpi_path,
- chrome_path, filename_suffix):
- record = (entry, locale_code, xpi_path, chrome_path, filename_suffix)
- self.traversal.append(record)
-
- def _processNestedJar(self, nested_recorder):
- self.traversal.append(nested_recorder.traversal)
-
- def _finish(self):
- self.traversal.append('.')
-
-
-class MozillaZipTraversalTestCase(unittest.TestCase):
- """Test Mozilla XPI/jar traversal."""
-
- layer = LaunchpadZopelessLayer
-
- def test_InvalidXpiFile(self):
- # If the "XPI" file isn't really a zip file, that's a
- # TranslationFormatInvalidInputError.
- self.assertRaises(
- TranslationFormatInvalidInputError,
- TraversalRecorder,
- 'foo.xpi', __file__)
-
- def test_XpiTraversal(self):
- """Test a typical traversal of XPI file, with nested jar file."""
- xpi_archive = get_en_US_xpi_file_to_import('en-US')
- record = TraversalRecorder('', xpi_archive)
- self.assertEqual(record.traversal, [
- [
- ('copyover1.foo', 'en-US',
- 'jar:chrome/en-US.jar!/copyover1.foo',
- 'main/copyover1.foo', '.foo'
- ),
- ('subdir/copyover2.foo', 'en-US',
- 'jar:chrome/en-US.jar!/subdir/copyover2.foo',
- 'main/subdir/copyover2.foo', '.foo'
- ),
- ('subdir/test2.dtd', 'en-US',
- 'jar:chrome/en-US.jar!/subdir/test2.dtd',
- 'main/subdir/test2.dtd', '.dtd'
- ),
- ('subdir/test2.properties', 'en-US',
- 'jar:chrome/en-US.jar!/subdir/test2.properties',
- 'main/subdir/test2.properties', '.properties'
- ),
- ('test1.dtd', 'en-US',
- 'jar:chrome/en-US.jar!/test1.dtd',
- 'main/test1.dtd', '.dtd'
- ),
- ('test1.properties', 'en-US',
- 'jar:chrome/en-US.jar!/test1.properties',
- 'main/test1.properties', '.properties'
- ),
- '.'
- ],
- '.'
- ])
-
- def test_XpiTraversalWithoutManifest(self):
- """Test traversal of an XPI file without manifest."""
- xpi_archive = get_en_US_xpi_file_to_import('no-manifest')
- record = TraversalRecorder('', xpi_archive)
- # Without manifest, there is no knowledge of locale or chrome
- # paths, so those are None.
- self.assertEqual(record.traversal, [
- [
- ('file.txt', None,
- 'jar:chrome/en-US.jar!/file.txt', None, '.txt'
- ),
- '.'
- ],
- ('no-jar.txt', None,
- 'no-jar.txt', None, '.txt'
- ),
- '.'
- ])
diff --git a/lib/lp/translations/utilities/tests/test_translation_importer.py b/lib/lp/translations/utilities/tests/test_translation_importer.py
index ce43cd8..dcf203b 100644
--- a/lib/lp/translations/utilities/tests/test_translation_importer.py
+++ b/lib/lp/translations/utilities/tests/test_translation_importer.py
@@ -72,9 +72,6 @@ class TranslationImporterTestCase(TestCaseWithFactory):
None,
importer.getTranslationFormatImporter(
TranslationFileFormat.KDEPO))
- self.assertIsNot(
- None,
- importer.getTranslationFormatImporter(TranslationFileFormat.XPI))
def testGetTranslationFileFormatByFileExtension(self):
"""Checked whether file format precedence works correctly."""
@@ -94,10 +91,6 @@ class TranslationImporterTestCase(TestCaseWithFactory):
importer.getTranslationFileFormat(
".po", BytesIO(b'msgid "_: kde context\nmessage"\nmsgstr ""')))
- self.assertEqual(
- TranslationFileFormat.XPI,
- importer.getTranslationFileFormat(".xpi", BytesIO(b"")))
-
def testNoConflictingPriorities(self):
"""Check that no two importers for the same file extension have
exactly the same priority."""
@@ -111,13 +104,12 @@ class TranslationImporterTestCase(TestCaseWithFactory):
def testFileExtensionsWithImporters(self):
"""Check whether we get the right list of file extensions handled."""
self.assertEqual(
- ['.po', '.pot', '.xpi'],
+ ['.po', '.pot'],
TranslationImporter().supported_file_extensions)
def testTemplateSuffixes(self):
"""Check for changes in filename suffixes that identify templates."""
- self.assertEqual(
- ['.pot', 'en-US.xpi'], TranslationImporter().template_suffixes)
+ self.assertEqual(['.pot'], TranslationImporter().template_suffixes)
def _assertIsNotTemplate(self, path):
self.assertFalse(
@@ -137,12 +129,8 @@ class TranslationImporterTestCase(TestCaseWithFactory):
self._assertIsTemplate("bar.pot")
self._assertIsTemplate("foo/bar.pot")
self._assertIsTemplate("foo.bar.pot")
- self._assertIsTemplate("en-US.xpi")
- self._assertIsTemplate("translations/en-US.xpi")
self._assertIsNotTemplate("pt_BR.po")
- self._assertIsNotTemplate("pt_BR.xpi")
- self._assertIsNotTemplate("pt-BR.xpi")
def testHiddenFilesRecognition(self):
# Hidden files and directories (leading dot) are recognized.
@@ -189,13 +177,9 @@ class TranslationImporterTestCase(TestCaseWithFactory):
self._assertIsTranslation("po/el.po")
self._assertIsTranslation("po/package-el.po")
self._assertIsTranslation("po/package-zh_TW.po")
- self._assertIsTranslation("en-GB.xpi")
- self._assertIsTranslation("translations/en-GB.xpi")
self._assertIsNotTranslation("hi.pot")
self._assertIsNotTranslation("po/hi.pot")
- self._assertIsNotTranslation("en-US.xpi")
- self._assertIsNotTranslation("translations/en-US.xpi")
def testIsIdenticalTranslation(self):
"""Test `is_identical_translation`."""
diff --git a/lib/lp/translations/utilities/tests/test_xpi_dtd_format.py b/lib/lp/translations/utilities/tests/test_xpi_dtd_format.py
deleted file mode 100644
index ddf0c70..0000000
--- a/lib/lp/translations/utilities/tests/test_xpi_dtd_format.py
+++ /dev/null
@@ -1,45 +0,0 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-__metaclass__ = type
-
-import unittest
-
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatInvalidInputError,
- )
-from lp.translations.utilities.mozilla_dtd_parser import DtdFile
-
-
-class DtdFormatTestCase(unittest.TestCase):
- """Test class for dtd file format."""
-
- def test_DtdSyntaxError(self):
- # Syntax errors in a DTD file are reported as translation format
- # errors.
- content = '<!ENTITY foo "gah"></ENTITY>'
- self.assertRaises(
- TranslationFormatInvalidInputError, DtdFile, 'test.dtd', None,
- content)
-
- def test_UTF8DtdFileTest(self):
- """This test makes sure that we handle UTF-8 encoding files."""
-
- content = (
- '<!ENTITY utf8.message "\xc2\xbfQuieres? \xc2\xa1S\xc3\xad!">')
-
- dtd_file = DtdFile('test.dtd', None, content)
-
- # There is a single message.
- self.assertEqual(len(dtd_file.messages), 1)
- message = dtd_file.messages[0]
-
- self.assertEqual([u'\xbfQuieres? \xa1S\xed!'], message.translations)
-
- def test_Latin1DtdFileTest(self):
- """This test makes sure that we detect bad encodings."""
-
- content = '<!ENTITY latin1.message "\xbfQuieres? \xa1S\xed!">\n'
-
- self.assertRaises(TranslationFormatInvalidInputError, DtdFile, None,
- 'test.dtd', content)
diff --git a/lib/lp/translations/utilities/tests/test_xpi_import.py b/lib/lp/translations/utilities/tests/test_xpi_import.py
deleted file mode 100644
index 1bbe439..0000000
--- a/lib/lp/translations/utilities/tests/test_xpi_import.py
+++ /dev/null
@@ -1,365 +0,0 @@
-# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Functional tests for XPI file format"""
-__metaclass__ = type
-
-import re
-import unittest
-
-from zope.component import getUtility
-
-from lp.app.interfaces.launchpad import ILaunchpadCelebrities
-from lp.registry.interfaces.person import IPersonSet
-from lp.registry.interfaces.product import IProductSet
-from lp.testing.layers import LaunchpadZopelessLayer
-from lp.translations.enums import RosettaImportStatus
-from lp.translations.interfaces.potemplate import IPOTemplateSet
-from lp.translations.utilities.mozilla_xpi_importer import MozillaXpiImporter
-from lp.translations.utilities.tests.helpers import (
- import_pofile_or_potemplate,
- )
-from lp.translations.utilities.tests.xpi_helpers import (
- access_key_source_comment,
- command_key_source_comment,
- get_en_US_xpi_file_to_import,
- )
-
-
-def unwrap(text):
- """Remove line breaks and any other wrapping artifacts from text."""
- return re.sub('\s+', ' ', text.strip())
-
-
-class XpiTestCase(unittest.TestCase):
- """XPI file import into Launchpad."""
-
- layer = LaunchpadZopelessLayer
-
- def setUp(self):
- # Get the importer.
- self.importer = getUtility(IPersonSet).getByName('mark')
-
- # Get the Firefox template.
- firefox_product = getUtility(IProductSet).getByName('firefox')
- firefox_productseries = firefox_product.getSeries('trunk')
- firefox_potemplate_subset = getUtility(IPOTemplateSet).getSubset(
- productseries=firefox_productseries)
- self.firefox_template = firefox_potemplate_subset.new(
- name='firefox',
- translation_domain='firefox',
- path='en-US.xpi',
- owner=self.importer)
- self.spanish_firefox = self.firefox_template.newPOFile('es')
- self.spanish_firefox.path = 'translations/es.xpi'
-
- def setUpTranslationImportQueueForTemplate(self, subdir):
- """Return an ITranslationImportQueueEntry for testing purposes.
-
- :param subdir: subdirectory in firefox-data to get XPI data from.
- """
- # Get the file to import.
- en_US_xpi = get_en_US_xpi_file_to_import(subdir)
- return import_pofile_or_potemplate(
- file_contents=en_US_xpi,
- person=self.importer,
- potemplate=self.firefox_template)
-
- def setUpTranslationImportQueueForTranslation(self, subdir):
- """Return an ITranslationImportQueueEntry for testing purposes.
-
- :param subdir: subdirectory in firefox-data to get XPI data from.
- """
- # Get the file to import. Given the way XPI file format works, we can
- # just use the same template file like a translation one.
- es_xpi = get_en_US_xpi_file_to_import(subdir)
- return import_pofile_or_potemplate(
- file_contents=es_xpi,
- person=self.importer,
- pofile=self.spanish_firefox,
- by_maintainer=True)
-
- def _assertXpiMessageInvariant(self, message):
- """Check whether invariant part of all messages are correct."""
- # msgid and singular_text are always different except for the keyboard
- # shortcuts which are the 'accesskey' and 'commandkey' ones.
- self.assertFalse(
- (message.msgid_singular.msgid == message.singular_text and
- message.msgid_singular.msgid not in (
- u'foozilla.menu.accesskey', u'foozilla.menu.commandkey')),
- 'msgid and singular_text should be different but both are %s' % (
- message.msgid_singular.msgid))
-
- # Plural forms should be None as this format is not able to handle
- # them.
- self.assertIsNone(message.msgid_plural)
- self.assertIsNone(message.plural_text)
-
- # There is no way to know whether a comment is from a
- # translator or a developer comment, so we have comenttext
- # always as None and store all comments as source comments.
- self.assertEqual(message.commenttext, u'')
-
- # This format doesn't support any functionality like .po flags.
- self.assertEqual(message.flagscomment, u'')
-
- def test_TemplateImport(self):
- """Test XPI template file import."""
- # Prepare the import queue to handle a new .xpi import.
- entry = self.setUpTranslationImportQueueForTemplate('en-US')
-
- # The status is now IMPORTED:
- self.assertEqual(entry.status, RosettaImportStatus.IMPORTED)
-
- # Let's validate the content of the messages.
- potmsgsets = list(self.firefox_template.getPOTMsgSets())
-
- messages_msgid_list = []
- for message in potmsgsets:
- messages_msgid_list.append(message.msgid_singular.msgid)
-
- # Check the common values for all messages.
- self._assertXpiMessageInvariant(message)
-
- if message.msgid_singular.msgid == u'foozilla.name':
- # It's a normal message that lacks any comment.
-
- self.assertEqual(message.singular_text, u'FooZilla!')
- self.assertEqual(
- message.filereferences,
- u'jar:chrome/en-US.jar!/test1.dtd(foozilla.name)')
- self.assertIsNone(message.sourcecomment)
-
- elif message.msgid_singular.msgid == u'foozilla.play.fire':
- # This one is also a normal message that has a comment.
-
- self.assertEqual(
- message.singular_text, u'Do you want to play with fire?')
- self.assertEqual(
- message.filereferences,
- u'jar:chrome/en-US.jar!/test1.dtd(foozilla.play.fire)')
- self.assertEqual(
- message.sourcecomment,
- u" Translators, don't play with fire! \n")
-
- elif message.msgid_singular.msgid == u'foozilla.utf8':
- # Now, we can see that special UTF-8 chars are extracted
- # correctly.
- self.assertEqual(
- message.singular_text, u'\u0414\u0430\u043d=Day')
- self.assertEqual(
- message.filereferences,
- u'jar:chrome/en-US.jar!/test1.properties:5' +
- u'(foozilla.utf8)')
- self.assertIsNone(message.sourcecomment)
- elif message.msgid_singular.msgid == u'foozilla.menu.accesskey':
- # access key is a special notation that is supposed to be
- # translated with a key shortcut.
- self.assertEqual(
- message.singular_text, u'M')
- self.assertEqual(
- message.filereferences,
- u'jar:chrome/en-US.jar!/subdir/test2.dtd' +
- u'(foozilla.menu.accesskey)')
- # The comment shows the key used when there is no translation,
- # which is noted as the en_US translation.
- self.assertEqual(
- unwrap(message.sourcecomment),
- unwrap(access_key_source_comment))
- elif message.msgid_singular.msgid == u'foozilla.menu.commandkey':
- # command key is a special notation that is supposed to be
- # translated with a key shortcut.
- self.assertEqual(
- message.singular_text, u'm')
- self.assertEqual(
- message.filereferences,
- u'jar:chrome/en-US.jar!/subdir/test2.dtd' +
- u'(foozilla.menu.commandkey)')
- # The comment shows the key used when there is no translation,
- # which is noted as the en_US translation.
- self.assertEqual(
- unwrap(message.sourcecomment),
- unwrap(command_key_source_comment))
-
- # Check that we got all messages.
- self.assertEqual(
- [u'foozilla.happytitle', u'foozilla.menu.accesskey',
- u'foozilla.menu.commandkey', u'foozilla.menu.title',
- u'foozilla.name', u'foozilla.nocomment', u'foozilla.play.fire',
- u'foozilla.play.ice', u'foozilla.title', u'foozilla.utf8',
- u'foozilla_something'],
- sorted(messages_msgid_list))
-
- def test_TwiceTemplateImport(self):
- """Test a template import done twice."""
- # Prepare the import queue to handle a new .xpi import.
- entry = self.setUpTranslationImportQueueForTemplate('en-US')
-
- # The status is now IMPORTED:
- self.assertEqual(entry.status, RosettaImportStatus.IMPORTED)
-
- # Retrieve the number of messages we got in this initial import.
- first_import_potmsgsets = self.firefox_template.getPOTMsgSets(
- ).count()
-
- # Force the entry to be imported again:
- entry.setStatus(RosettaImportStatus.APPROVED,
- getUtility(ILaunchpadCelebrities).rosetta_experts)
- # Now, we tell the PO template to import from the file data it has.
- (subject, body) = self.firefox_template.importFromQueue(entry)
-
- # Retrieve the number of messages we got in this second import.
- second_import_potmsgsets = self.firefox_template.getPOTMsgSets(
- ).count()
-
- # Both must match.
- self.assertEqual(first_import_potmsgsets, second_import_potmsgsets)
-
- def test_TranslationImport(self):
- """Test XPI translation file import."""
- # Prepare the import queue to handle a new .xpi import.
- template_entry = self.setUpTranslationImportQueueForTemplate('en-US')
- translation_entry = self.setUpTranslationImportQueueForTranslation(
- 'en-US')
-
- # The status is now IMPORTED:
- self.assertEqual(
- translation_entry.status, RosettaImportStatus.IMPORTED)
- self.assertEqual(template_entry.status, RosettaImportStatus.IMPORTED)
-
- # Let's validate the content of the messages.
- potmsgsets = list(self.firefox_template.getPOTMsgSets())
-
- messages = [message.msgid_singular.msgid for message in potmsgsets]
- messages.sort()
- self.assertEqual(
- [u'foozilla.happytitle',
- u'foozilla.menu.accesskey',
- u'foozilla.menu.commandkey',
- u'foozilla.menu.title',
- u'foozilla.name',
- u'foozilla.nocomment',
- u'foozilla.play.fire',
- u'foozilla.play.ice',
- u'foozilla.title',
- u'foozilla.utf8',
- u'foozilla_something'],
- messages)
-
- potmsgset = self.firefox_template.getPOTMsgSetByMsgIDText(
- u'foozilla.name', context='main/test1.dtd')
- translation = potmsgset.getCurrentTranslation(
- self.firefox_template, self.spanish_firefox.language,
- self.firefox_template.translation_side)
-
- # It's a normal message that lacks any comment.
- self.assertEqual(potmsgset.singular_text, u'FooZilla!')
-
- # With this first import, upstream and Ubuntu translations must match.
- self.assertEqual(
- translation.translations,
- potmsgset.getOtherTranslation(
- self.spanish_firefox.language,
- self.firefox_template.translation_side).translations)
-
- potmsgset = self.firefox_template.getPOTMsgSetByMsgIDText(
- u'foozilla.menu.accesskey', context='main/subdir/test2.dtd')
-
- # access key is a special notation that is supposed to be
- # translated with a key shortcut.
- self.assertEqual(potmsgset.singular_text, u'M')
- # The comment shows the key used when there is no translation,
- # which is noted as the en_US translation.
- self.assertEqual(
- unwrap(potmsgset.sourcecomment),
- unwrap(access_key_source_comment))
- # But for the translation import, we get the key directly.
- self.assertEqual(
- potmsgset.getOtherTranslation(
- self.spanish_firefox.language,
- self.firefox_template.translation_side).translations,
- [u'M'])
-
- potmsgset = self.firefox_template.getPOTMsgSetByMsgIDText(
- u'foozilla.menu.commandkey', context='main/subdir/test2.dtd')
- # command key is a special notation that is supposed to be
- # translated with a key shortcut.
- self.assertEqual(
- potmsgset.singular_text, u'm')
- # The comment shows the key used when there is no translation,
- # which is noted as the en_US translation.
- self.assertEqual(
- unwrap(potmsgset.sourcecomment),
- unwrap(command_key_source_comment))
- # But for the translation import, we get the key directly.
- self.assertEqual(
- potmsgset.getOtherTranslation(
- self.spanish_firefox.language,
- self.firefox_template.translation_side).translations,
- [u'm'])
-
- def test_GetLastTranslator(self):
- """Tests whether we extract last translator information correctly."""
- translation_entry = self.setUpTranslationImportQueueForTranslation(
- 'en-US')
- importer = MozillaXpiImporter()
- translation_file = importer.parse(translation_entry)
-
- # Let's try with the translation file, it has valid Last Translator
- # information.
- name, email = translation_file.header.getLastTranslator()
- self.assertEqual(name, u'Carlos Perell\xf3 Mar\xedn')
- self.assertEqual(email, u'carlos@xxxxxxxxxxxxx')
-
- def test_Contexts(self):
- """Test that message context in XPI file is set to chrome path."""
- queue_entry = self.setUpTranslationImportQueueForTranslation(
- 'clashing_ids')
- importer = MozillaXpiImporter()
- template = importer.parse(queue_entry)
-
- messages = sorted([
- (message.msgid_singular, message.context, message.singular_text)
- for message in template.messages])
- self.assertEqual(
- [
- (u'foozilla.clashing.key',
- u'mac/extra.dtd',
- u'This message is Mac-specific, and comes from DTD.'),
- (u'foozilla.clashing.key',
- u'mac/extra.properties',
- u'This message is Mac-specific, and comes from properties.'),
- (u'foozilla.clashing.key',
- u'main/main.dtd',
- u'This message is in the main DTD.'),
- (u'foozilla.clashing.key',
- u'main/main.properties',
- u'This message is in the main properties file.'),
- (u'foozilla.clashing.key',
- u'unix/extra.dtd',
- u'This message is Unix-specific, and comes from DTD.'),
- (u'foozilla.clashing.key',
- u'unix/extra.properties',
- u'This message is Unix-specific, and comes from properties.'),
- (u'foozilla.clashing.key',
- u'win/extra.dtd',
- u'This message is Windows-specific, and comes from DTD.'),
- (u'foozilla.clashing.key',
- u'win/extra.properties',
- u'This message is Windows-specific, '
- 'and comes from properties.'),
- (u'foozilla.regular.message',
- u'main/main.dtd',
- u'A non-clashing message.'),
- ],
- messages)
-
- def test_SystemEntityIsIgnored(self):
- """Test handling of SYSTEM entities in DTD files."""
- self.setUpTranslationImportQueueForTemplate('system-entity')
- msgids = [
- (potmsgset.msgid_singular.msgid, potmsgset.singular_text)
- for potmsgset in self.firefox_template.getPOTMsgSets()]
- self.assertEqual(msgids, [
- ('firststring', 'First translatable string'),
- ('secondstring', 'Second translatable string')])
diff --git a/lib/lp/translations/utilities/tests/test_xpi_manifest.py b/lib/lp/translations/utilities/tests/test_xpi_manifest.py
deleted file mode 100644
index b5348df..0000000
--- a/lib/lp/translations/utilities/tests/test_xpi_manifest.py
+++ /dev/null
@@ -1,307 +0,0 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Unit tests for XPI manifests."""
-
-__metaclass__ = type
-
-import unittest
-
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatSyntaxError,
- )
-from lp.translations.utilities.xpi_manifest import XpiManifest
-
-
-class XpiManifestTestCase(unittest.TestCase):
- """Test `XpiManifest`."""
-
- def test_TrivialParse(self):
- # Parse and use minimal manifest.
- manifest = XpiManifest("locale chromepath en-US directory/")
- self.assertEqual(len(manifest._locales), 1)
- chrome_path, locale = manifest.getChromePathAndLocale(
- 'directory/file.dtd')
- self.assertIsNotNone(chrome_path, "Failed to match simple path")
- self.assertEqual(
- chrome_path, "chromepath/file.dtd", "Bad chrome path")
-
- def test_NonMatch(self):
- # Failure to match path.
- manifest = XpiManifest("locale chromepath en-US directory/")
- chrome_path, locale = manifest.getChromePathAndLocale(
- 'nonexistent/file')
- self.assertIsNone(chrome_path, "Unexpected path match.")
- self.assertIsNone(locale, "Got locale without a match.")
-
- def test_NoUsefulLines(self):
- # Parse manifest without useful data. Lines that don't match what
- # we're looking for are ignored.
- manifest = XpiManifest("""
- There are no usable
- locale lines
- in this file.
- """.lstrip())
- self.assertEqual(len(manifest._locales), 0)
- chrome_path, locale = manifest.getChromePathAndLocale('lines')
- self.assertIsNone(chrome_path, "Empty manifest matched a path.")
- chrome_path, locale = manifest.getChromePathAndLocale('')
- self.assertIsNone(chrome_path, "Matched empty path.")
-
- def _checkSortOrder(self, manifest):
- """Verify that manifest is sorted by increasing path length."""
- last_entry = None
- for entry in manifest._locales:
- if last_entry is not None:
- self.assertFalse(len(entry.path) < len(last_entry.path),
- "Manifest entries not sorted by increasing path length.")
- last_entry = entry
-
- def test_MultipleLines(self):
- # Parse manifest file with multiple entries.
- manifest = XpiManifest("""
- locale foo en-US foodir/
- locale bar en-US bardir/
- locale ixx en-US ixxdir/
- locale gna en-US gnadir/
- """.lstrip())
- self.assertEqual(len(manifest._locales), 4)
- self._checkSortOrder(manifest)
- for dir in ['gna', 'bar', 'ixx', 'foo']:
- path = "%sdir/file.html" % dir
- chrome_path, locale = manifest.getChromePathAndLocale(path)
- self.assertEqual(chrome_path, "%s/file.html" % dir,
- "Bad chrome path in multi-line parse.")
- self.assertEqual(
- locale, 'en-US', "Bad locale in multi-line parse.")
-
- def test_MultipleLocales(self):
- # Different locales.
- dirs = {
- 'foo': 'en-US',
- 'bar': 'es',
- 'ixx': 'zh_CN',
- 'zup': 'zh_TW',
- 'gna': 'pt',
- 'gnu': 'pt_BR'
- }
- manifest_text = '\n'.join([
- "locale %s %s %sdir/\n" % (dir, locale, dir)
- for dir, locale in dirs.iteritems()
- ])
- manifest = XpiManifest(manifest_text)
- self._checkSortOrder(manifest)
- for dir, dirlocale in dirs.iteritems():
- path = "%sdir/file.html" % dir
- chrome_path, locale = manifest.getChromePathAndLocale(path)
- self.assertEqual(chrome_path, "%s/file.html" % dir,
- "Bad chrome path in multi-line parse.")
- self.assertEqual(locale, dirlocale, "Locales got mixed up.")
-
- def test_IgnoredLines(self):
- # Ignored lines: anything that doesn't start with "locale" or doesn't
- # have the right number of arguments. The one correct line is picked
- # out though.
- manifest = XpiManifest("""
- nonlocale obsolete fr foodir/
- anotherline
-
- #locale obsolete fr foodir/
- locale okay fr foodir/
- locale overlong fr foordir/ etc. etc. etc.
- locale incomplete fr
- """.lstrip())
- self.assertEqual(len(manifest._locales), 1)
- chrome_path, locale = manifest.getChromePathAndLocale('foodir/x')
- self.assertIsNotNone(chrome_path, "Garbage lines messed up match.")
- self.assertEqual(chrome_path, "okay/x", "Matched wrong line.")
- self.assertEqual(locale, "fr", "Inexplicably mismatched locale.")
-
- def test_DuplicateLines(self):
- # The manifest ignores redundant lines with the same path.
- manifest = XpiManifest("""
- locale dup fy boppe
- locale dup fy boppe
- """.lstrip())
- self.assertEqual(len(manifest._locales), 1)
-
- def _checkLookup(self, manifest, path, chrome_path, locale):
- """Helper: look up `path` in `manifest`, expect given output."""
- found_chrome_path, found_locale = manifest.getChromePathAndLocale(
- path)
- self.assertIsNotNone(found_chrome_path, "No match found for " + path)
- self.assertEqual(found_chrome_path, chrome_path)
- self.assertEqual(found_locale, locale)
-
- def test_NormalizedLookup(self):
- # Both sides of a path lookup are normalized, so that a matching
- # prefix is recognized in a path even if the two have some meaningless
- # differences in their spelling.
- manifest = XpiManifest("locale x nn //a/dir")
- self._checkLookup(manifest, "a//dir///etc", 'x/etc', 'nn')
-
- def _checkNormalize(self, bad_path, good_path):
- """Test that `bad_path` normalizes to `good_path`."""
- self.assertEqual(XpiManifest._normalizePath(bad_path), good_path)
-
- def test_Normalize(self):
- # These paths are all wrong or difficult for one reason or another.
- # Check that the normalization of paths renders those little
- # imperfections irrelevant to path lookup.
- self._checkNormalize('x/', 'x/')
- self._checkNormalize('x', 'x')
- self._checkNormalize('/x', 'x')
- self._checkNormalize('//x', 'x')
- self._checkNormalize('/x/', 'x/')
- self._checkNormalize('x//', 'x/')
- self._checkNormalize('x///', 'x/')
- self._checkNormalize('x/y/', 'x/y/')
- self._checkNormalize('x/y', 'x/y')
- self._checkNormalize('x//y/', 'x/y/')
-
- def test_PathBoundaries(self):
- # Paths can only match on path boundaries, where the slashes are
- # supposed to be.
- manifest = XpiManifest("""
- locale short el /ploink/squit
- locale long he /ploink/squittle
- """.lstrip())
- self._checkSortOrder(manifest)
- self._checkLookup(manifest, 'ploink/squit/x', 'short/x', 'el')
- self._checkLookup(manifest, '/ploink/squittle/x', 'long/x', 'he')
-
- def test_Overlap(self):
- # Path matching looks for longest prefix. Make sure this works right,
- # even when nested directories are in "overlapping" manifest entries.
- manifest = XpiManifest("""
- locale foo1 ca a/
- locale foo2 ca a/b/
- locale foo3 ca a/b/c/x1
- locale foo4 ca a/b/c/x2
- """.lstrip())
- self._checkSortOrder(manifest)
- self._checkLookup(manifest, 'a/bb', 'foo1/bb', 'ca')
- self._checkLookup(manifest, 'a/bb/c', 'foo1/bb/c', 'ca')
- self._checkLookup(manifest, 'a/b/y', 'foo2/y', 'ca')
- self._checkLookup(manifest, 'a/b/c/', 'foo2/c/', 'ca')
- self._checkLookup(manifest, 'a/b/c/x12', 'foo2/c/x12', 'ca')
- self._checkLookup(manifest, 'a/b/c/x1/y', 'foo3/y', 'ca')
- self._checkLookup(manifest, 'a/b/c/x2/y', 'foo4/y', 'ca')
-
- def test_JarLookup(self):
- # Simple, successful lookup of a correct path inside a jar file.
- manifest = XpiManifest("""
- locale foo en_GB jar:foo.jar!/dir/
- locale bar id jar:bar.jar!/
- """.lstrip())
- self._checkSortOrder(manifest)
- self._checkLookup(
- manifest, 'jar:foo.jar!/dir/file', 'foo/file', 'en_GB')
- self._checkLookup(
- manifest, 'jar:bar.jar!/dir/file', 'bar/dir/file', 'id')
-
- def test_JarNormalization(self):
- # Various badly-formed or corner-case paths. All get normalized.
- self._checkNormalize('jar:jarless/path', 'jarless/path')
- self._checkNormalize(
- 'jar:foo.jar!/contained/file', 'jar:foo.jar!/contained/file')
- self._checkNormalize(
- 'foo.jar!contained/file', 'jar:foo.jar!/contained/file')
- self._checkNormalize(
- 'jar:foo.jar!//contained/file', 'jar:foo.jar!/contained/file')
- self._checkNormalize('splat.jar!', 'jar:splat.jar!/')
- self._checkNormalize('dir/x.jar!dir', 'jar:dir/x.jar!/dir')
-
- def test_NestedJarNormalization(self):
- # Test that paths with jars inside jars are normalized correctly.
- self._checkNormalize(
- 'jar:dir/x.jar!/y.jar!/dir', 'jar:dir/x.jar!/y.jar!/dir')
- self._checkNormalize(
- 'dir/x.jar!y.jar!dir', 'jar:dir/x.jar!/y.jar!/dir')
- self._checkNormalize(
- 'dir/x.jar!/dir/y.jar!', 'jar:dir/x.jar!/dir/y.jar!/')
-
- def test_JarMixup(self):
- # Two jar files can have files for the same locale. Two locales can
- # have files in the same jar file. Two translations in different
- # places can have the same chrome path.
- manifest = XpiManifest("""
- locale serbian sr jar:translations.jar!/sr/
- locale croatian hr jar:translations.jar!/hr/
- locale docs sr jar:docs.jar!/sr/
- locale docs hr jar:docs.jar!/hr/
- """.lstrip())
- self._checkSortOrder(manifest)
- self._checkLookup(
- manifest, 'jar:translations.jar!/sr/x', 'serbian/x', 'sr')
- self._checkLookup(
- manifest, 'jar:translations.jar!/hr/x', 'croatian/x', 'hr')
- self._checkLookup(manifest, 'jar:docs.jar!/sr/x', 'docs/x', 'sr')
- self._checkLookup(manifest, 'jar:docs.jar!/hr/x', 'docs/x', 'hr')
-
- def test_NestedJars(self):
- # Jar files can be contained in jar files.
- manifest = XpiManifest("""
- locale x it jar:dir/x.jar!/subdir/y.jar!/
- locale y it jar:dir/x.jar!/subdir/y.jar!/deep/
- locale z it jar:dir/x.jar!/subdir/z.jar!/
- """.lstrip())
- self._checkSortOrder(manifest)
- self._checkLookup(
- manifest, 'jar:dir/x.jar!/subdir/y.jar!/foo', 'x/foo', 'it')
- self._checkLookup(
- manifest, 'jar:dir/x.jar!/subdir/y.jar!/deep/foo', 'y/foo', 'it')
- self._checkLookup(
- manifest, 'dir/x.jar!/subdir/z.jar!/foo', 'z/foo', 'it')
-
- def test_ContainsLocales(self):
- # Jar files need to be descended into if any locale line mentions a
- # path inside them.
- manifest = XpiManifest("locale in my jar:x/foo.jar!/y")
- self.assertTrue(manifest.containsLocales("jar:x/foo.jar!/"))
- self.assertFalse(manifest.containsLocales("jar:zzz/foo.jar!/"))
-
- def test_NormalizeContainsLocales(self):
- # "containsLocales" lookup is normalized, just like chrome path
- # lookup, so it's not fazed by syntactical misspellings.
- manifest = XpiManifest("locale main kh jar:/x/foo.jar!bar.jar!")
- self.assertTrue(manifest.containsLocales("x/foo.jar!//bar.jar!/"))
-
- def test_ReverseMapping(self):
- # Test "reverse mapping" from chrome path to XPI path.
- manifest = XpiManifest(
- "locale browser en-US jar:locales/en-US.jar!/chrome/")
- path = manifest.findMatchingXpiPath('browser/gui/print.dtd', 'en-US')
- self.assertEqual(path, "jar:locales/en-US.jar!/chrome/gui/print.dtd")
-
- def test_NoReverseMapping(self):
- # Failed reverse lookup.
- manifest = XpiManifest(
- "locale browser en-US jar:locales/en-US.jar!/chrome/")
- path = manifest.findMatchingXpiPath('manual/gui/print.dtd', 'en-US')
- self.assertEqual(path, None)
-
- def test_ReverseMappingWrongLocale(self):
- # Reverse mapping fails if given the wrong locale.
- manifest = XpiManifest(
- "locale browser en-US jar:locales/en-US.jar!/chrome/")
- path = manifest.findMatchingXpiPath('browser/gui/print.dtd', 'pt')
- self.assertEqual(path, None)
-
- def test_ReverseMappingLongestMatch(self):
- # Reverse mapping always finds the longest match.
- manifest = XpiManifest("""
- locale browser en-US jar:locales/
- locale browser en-US jar:locales/en-US.jar!/chrome/
- locale browser en-US jar:locales/en-US.jar!/
- """.lstrip())
- path = manifest.findMatchingXpiPath('browser/gui/print.dtd', 'en-US')
- self.assertEqual(path, "jar:locales/en-US.jar!/chrome/gui/print.dtd")
-
- def test_blank_line(self):
- # Manifests must not begin with newline.
- self.assertRaises(
- TranslationFormatSyntaxError,
- XpiManifest, """
- locale browser en-US jar:locales
- """)
diff --git a/lib/lp/translations/utilities/tests/test_xpi_po_exporter.py b/lib/lp/translations/utilities/tests/test_xpi_po_exporter.py
index 532f9bf..ca830c7 100644
--- a/lib/lp/translations/utilities/tests/test_xpi_po_exporter.py
+++ b/lib/lp/translations/utilities/tests/test_xpi_po_exporter.py
@@ -1,21 +1,29 @@
+# -*- coding: utf-8 -*-
+# NOTE: The first line above must stay first; do not move the copyright
+# notice to the top. See http://www.python.org/dev/peps/pep-0263/.
+#
# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
# GNU Affero General Public License version 3 (see the file LICENSE).
__metaclass__ = type
from textwrap import dedent
-import unittest
+from fixtures import MonkeyPatch
import transaction
from zope.component import (
getAdapter,
getUtility,
)
+from zope.interface import implementer
from zope.interface.verify import verifyObject
+from zope.security.proxy import removeSecurityProxy
from lp.app.interfaces.launchpad import ILaunchpadCelebrities
from lp.registry.interfaces.person import IPersonSet
from lp.registry.interfaces.product import IProductSet
+from lp.testing import TestCase
+from lp.testing.fixture import ZopeUtilityFixture
from lp.testing.layers import LaunchpadZopelessLayer
from lp.translations.enums import RosettaImportStatus
from lp.translations.interfaces.potemplate import IPOTemplateSet
@@ -25,21 +33,165 @@ from lp.translations.interfaces.translationcommonformat import (
from lp.translations.interfaces.translationexporter import (
ITranslationFormatExporter,
)
+from lp.translations.interfaces.translationfileformat import (
+ TranslationFileFormat,
+ )
+from lp.translations.interfaces.translationimporter import (
+ ITranslationFormatImporter,
+ ITranslationImporter,
+ )
from lp.translations.interfaces.translationimportqueue import (
ITranslationImportQueue,
)
-from lp.translations.utilities.tests.test_xpi_import import (
- get_en_US_xpi_file_to_import,
+from lp.translations.interfaces.translations import TranslationConstants
+from lp.translations.utilities.translation_common_format import (
+ TranslationFileData,
+ TranslationMessageData,
)
from lp.translations.utilities.translation_export import ExportFileStorage
+from lp.translations.utilities.xpi_header import XpiHeader
from lp.translations.utilities.xpi_po_exporter import XPIPOExporter
-class XPIPOExporterTestCase(unittest.TestCase):
+# Hardcoded representations of what used to be found in
+# lib/lp/translations/utilities/tests/firefox-data/. We no longer have real
+# XPI import code, so we pre-parse the messages.
+test_xpi_header = dedent(u'''\
+ <?xml version="1.0"?>
+ <RDF xmlns="http://www.w3.org/1999/02/22-rdf-syntax-ns#";
+ xmlns:em="http://www.mozilla.org/2004/em-rdf#";>
+ <Description about="urn:mozilla:install-manifest"
+ em:id="langpack-en-US@xxxxxxxxxxxxxxxxxxx"
+ em:name="English U.S. (en-US) Language Pack"
+ em:version="2.0"
+ em:type="8"
+ em:creator="Danilo Šegan">
+ <em:contributor>Данило Шеган</em:contributor>
+ <em:contributor>Carlos Perelló Marín <carlos@xxxxxxxxxxxxx></em:contributor>
+
+ <em:targetApplication>
+ <Description>
+ <em:id>{ec8030f7-c20a-464f-9b0e-13a3a9e97384}</em:id><!-- firefox -->
+ <em:minVersion>2.0</em:minVersion>
+ <em:maxVersion>2.0.0.*</em:maxVersion>
+ </Description>
+ </em:targetApplication>
+ </Description>
+ </RDF>
+''')
+test_xpi_messages = [
+ (u'foozilla.menu.title', u'main/subdir/test2.dtd',
+ u'jar:chrome/en-US.jar!/subdir/test2.dtd', u'MENU',
+ u' This is a DTD file inside a subdirectory\n'),
+ (u'foozilla.menu.accesskey', u'main/subdir/test2.dtd',
+ u'jar:chrome/en-US.jar!/subdir/test2.dtd', u'M',
+ dedent(u'''\
+ Select the access key that you want to use. These have
+ to be translated in a way that the selected character is
+ present in the translated string of the label being
+ referred to, for example 'i' in 'Edit' menu item in
+ English. If a translation already exists, please don't
+ change it if you are not sure about it. Please find the
+ context of the key from the end of the 'Located in' text
+ below.
+ ''')),
+ (u'foozilla.menu.commandkey', u'main/subdir/test2.dtd',
+ u'jar:chrome/en-US.jar!/subdir/test2.dtd', u'm',
+ dedent(u'''\
+ Select the shortcut key that you want to use. It
+ should be translated, but often shortcut keys (for
+ example Ctrl + KEY) are not changed from the original. If
+ a translation already exists, please don't change it if
+ you are not sure about it. Please find the context of
+ the key from the end of the 'Located in' text below.
+ ''')),
+ (u'foozilla_something', u'main/subdir/test2.properties',
+ u'jar:chrome/en-US.jar!/subdir/test2.properties:6', u'SomeZilla',
+ dedent(u'''\
+ Translators, what you are seeing now is a lovely,
+ awesome, multiline comment aimed at you directly
+ from the streets of a .properties file
+ ''')),
+ (u'foozilla.name', u'main/test1.dtd',
+ u'jar:chrome/en-US.jar!/test1.dtd', u'FooZilla!', None),
+ (u'foozilla.play.fire', u'main/test1.dtd',
+ u'jar:chrome/en-US.jar!/test1.dtd', u'Do you want to play with fire?',
+ u" Translators, don't play with fire!\n"),
+ (u'foozilla.play.ice', u'main/test1.dtd',
+ u'jar:chrome/en-US.jar!/test1.dtd', u'Play with ice?',
+ u' This is just a comment, not a comment for translators\n'),
+ (u'foozilla.title', u'main/test1.properties',
+ u'jar:chrome/en-US.jar!/test1.properties:1', u'FooZilla Zilla Thingy',
+ None),
+ (u'foozilla.happytitle', u'main/test1.properties',
+ u'jar:chrome/en-US.jar!/test1.properties:3',
+ u'http://foozillingy.happy.net/',
+ u"Translators, if you're older than six, don't translate this\n"),
+ (u'foozilla.nocomment', u'main/test1.properties',
+ u'jar:chrome/en-US.jar!/test1.properties:4', u'No Comment',
+ u'(Except this one)\n'),
+ (u'foozilla.utf8', u'main/test1.properties',
+ u'jar:chrome/en-US.jar!/test1.properties:5', u'\u0414\u0430\u043d=Day',
+ None),
+ ]
+
+
+class FakeXPIMessage(TranslationMessageData):
+ """Simulate an XPI translation message."""
+
+ def __init__(self, key, chrome_path, file_and_line, value, last_comment):
+ super(FakeXPIMessage, self).__init__()
+ self.msgid_singular = key
+ self.context = chrome_path
+ self.file_references = '%s(%s)' % (file_and_line, key)
+ value = value.strip()
+ self.addTranslation(TranslationConstants.SINGULAR_FORM, value)
+ self.singular_text = value
+ self.source_comment = last_comment
+
+
+@implementer(ITranslationFormatImporter)
+class FakeXPIImporter:
+ """Simulate an XPI import. We no longer have real XPI import code."""
+
+ def getFormat(self, file_contents):
+ """See `ITranslationFormatImporter`."""
+ return TranslationFileFormat.XPI
+
+ priority = 0
+ content_type = 'application/zip'
+ file_extensions = ['.xpi']
+ template_suffix = 'en-US.xpi'
+ uses_source_string_msgids = True
+
+ def parse(self, translation_import_queue_entry):
+ """See `ITranslationFormatImporter`.
+
+ This takes a `TranslationImportQueueEntry` to satisfy the interface,
+ but ignores it and returns hardcoded data instead.
+ """
+ translation_file = TranslationFileData()
+ translation_file.header = XpiHeader(test_xpi_header)
+ translation_file.messages = [
+ FakeXPIMessage(*message) for message in test_xpi_messages]
+ return translation_file
+
+ def getHeaderFromString(self, header_string):
+ """See `ITranslationFormatImporter`.
+
+ This takes a header string to satisfy the interface, but ignores it
+ and returns hardcoded data instead.
+ """
+ return XpiHeader(test_xpi_header)
+
+
+class XPIPOExporterTestCase(TestCase):
"""Class test for gettext's .po file exports"""
layer = LaunchpadZopelessLayer
def setUp(self):
+ super(XPIPOExporterTestCase, self).setUp()
+
self.translation_exporter = XPIPOExporter()
# Get the importer.
@@ -75,14 +227,24 @@ class XPIPOExporterTestCase(unittest.TestCase):
def setUpTranslationImportQueueForTemplate(self):
"""Return an ITranslationImportQueueEntry for testing purposes."""
- # Get the file to import.
- en_US_xpi = get_en_US_xpi_file_to_import('en-US')
+ # Install a fake XPI importer, since we no longer have real XPI
+ # import code.
+ fake_xpi_importer = FakeXPIImporter()
+ self.useFixture(MonkeyPatch(
+ 'lp.translations.utilities.translation_import.importers',
+ {TranslationFileFormat.XPI: fake_xpi_importer}))
+ # Temporarily reinstall the translation importer without a security
+ # proxy. This avoids problems getting attributes of
+ # FakeXPIImporter, which has no Zope permissions defined.
+ self.useFixture(ZopeUtilityFixture(
+ removeSecurityProxy(getUtility(ITranslationImporter)),
+ ITranslationImporter))
# Attach it to the import queue.
translation_import_queue = getUtility(ITranslationImportQueue)
by_maintainer = True
entry = translation_import_queue.addOrUpdateEntry(
- self.firefox_template.path, en_US_xpi, by_maintainer,
+ self.firefox_template.path, b'dummy', by_maintainer,
self.importer, productseries=self.firefox_template.productseries,
potemplate=self.firefox_template)
diff --git a/lib/lp/translations/utilities/tests/test_xpi_properties_format.py b/lib/lp/translations/utilities/tests/test_xpi_properties_format.py
deleted file mode 100644
index 1d42b14..0000000
--- a/lib/lp/translations/utilities/tests/test_xpi_properties_format.py
+++ /dev/null
@@ -1,302 +0,0 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-__metaclass__ = type
-
-from textwrap import dedent
-import unittest
-
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatInvalidInputError,
- )
-from lp.translations.utilities.mozilla_xpi_importer import PropertyFile
-from lp.translations.utilities.xpi_properties_exporter import (
- XpiPropertiesSubExporter,
- )
-
-
-class PropertyFileFormatTestCase(unittest.TestCase):
- """Test class for property file format."""
-
- def _baseContentEncodingTest(self, content):
- """This is a base function to check different encodings."""
- property_file = PropertyFile('test.properties', None, dedent(content))
-
- expected = {u'default-first-title-mac': [u'Introducci\xf3n'],
- u'default-last-title-mac': [u'Conclusi\xf3n']}
- parsed = dict([(message.msgid_singular, message.translations)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_UTF8PropertyFileTest(self):
- """This test makes sure that we handle UTF-8 encoding files."""
- content = '''
- default-first-title-mac = Introducci\xc3\xb3n
- default-last-title-mac = Conclusi\xc3\xb3n
- '''
- self._baseContentEncodingTest(content)
-
- def test_UnicodeEscapedPropertyFileTest(self):
- """This test makes sure that we handle unicode escaped files."""
- content = '''
- default-first-title-mac=Introducci\u00F3n
- default-last-title-mac=Conclusi\u00F3n
- '''
- self._baseContentEncodingTest(content)
-
- def test_InvalidPropertyFileUnicodeEscape(self):
- # An invalid Unicode escape sequence is a
- # TranslationFormatInvalidInputError.
- content = '''
- weirdness=\u1
- '''
- self.assertRaises(
- TranslationFormatInvalidInputError, PropertyFile, None,
- 'test.properties', content)
-
- def test_Latin1PropertyFileTest(self):
- """This test makes sure that we detect bad encodings."""
- content = '''
- default-first-title-mac = Introducci\xf3n
- default-last-title-mac = Conclusi\xf3n
- '''
- self.assertRaises(
- TranslationFormatInvalidInputError, PropertyFile, None,
- 'test.properties', content)
-
- def test_TrailingBackslashPropertyFileTest(self):
- """Test whether trailing backslashes are well handled.
-
- A trailing backslash as last char in the line continue the string in
- the following document line.
- """
- content = '''
-default-first-title-mac=Introd\
-ucci\u00F3n
-'''
- property_file = PropertyFile('test.properties', None, dedent(content))
-
- expected = {u'default-first-title-mac': [u'Introducci\xf3n']}
- parsed = dict([(message.msgid_singular, message.translations)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_EscapedQuotesPropertyFileTest(self):
- """Test whether escaped quotes are well handled.
-
- Escaped quotes must be stored unescaped.
- """
- content = 'default-first-title-mac = \\\'Something\\\' \\\"more\\\"'
-
- property_file = PropertyFile('test.properties', None, dedent(content))
-
- expected = {u'default-first-title-mac': [u'\'Something\' \"more\"']}
- parsed = dict([(message.msgid_singular, message.translations)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_WholeLineCommentPropertyFileTest(self):
- """Test whether whole line comments are well handled."""
- content = '''
- # Foo bar comment.
- default-first-title-mac = blah
-
- # This comment should be ignored.
-
- foo = bar
- '''
-
- property_file = PropertyFile('test.properties', None, dedent(content))
- expected = {u'default-first-title-mac': u'Foo bar comment.\n',
- u'foo': None}
- parsed = dict([(message.msgid_singular, message.source_comment)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_EndOfLineCommentPropertyFileTest(self):
- """Test whether end of line comments are well handled."""
-
- content = '''
- default-first-title-mac = blah // Foo bar comment.
-
- # This comment should be ignored.
- foo = bar // Something
- '''
-
- property_file = PropertyFile('test.properties', None, dedent(content))
- expected_comments = {
- u'default-first-title-mac': u'Foo bar comment.\n',
- u'foo': u'Something\n'
- }
- parsed_comments = dict(
- [(message.msgid_singular, message.source_comment)
- for message in property_file.messages])
-
- self.assertEqual(expected_comments, parsed_comments)
-
- expected_translations = {
- u'default-first-title-mac': [u'blah'],
- u'foo': [u'bar']
- }
- parsed_translations = dict([(message.msgid_singular,
- message.translations)
- for message in property_file.messages])
-
- self.assertEqual(expected_translations, parsed_translations)
-
- def test_MultiLineCommentPropertyFileTest(self):
- """Test whether multiline comments are well handled."""
- content = '''
- /* single line comment */
- default-first-title-mac = blah
-
- /* Multi line comment
- yeah, it's multiple! */
- foo = bar
-
- /* Even with nested comment tags, we handle this as multiline comment:
- # fooo
- foos = bar
- something = else // Comment me!
- */
- long_comment = foo
- '''
-
- property_file = PropertyFile('test.properties', None, dedent(content))
- expected = {
- u'default-first-title-mac': u' single line comment \n',
- u'foo': u" Multi line comment\n yeah, it's multiple! \n",
- u'long_comment': (
- u' Even with nested comment tags, we handle this as' +
- u' multiline comment:\n# fooo\nfoos = bar\n' +
- u'something = else // Comment me!\n')
- }
- parsed = dict([(message.msgid_singular, message.source_comment)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_URLNotComment(self):
- """Double slash in a URL is not treated as end-of-line comment."""
- content = '''
- url = https://admin.example.com/ // Double slash in URL!
- '''
- property_file = PropertyFile('test.properties', None, dedent(content))
- message = None
- for entry in property_file.messages:
- self.assertEqual(message, None, "More messages than expected.")
- message = entry
-
- self.assertEqual(message.msgid_singular, u"url")
- self.assertEqual(message.singular_text, u"https://admin.example.com/";)
- self.assertEqual(message.source_comment, u"Double slash in URL!\n")
-
- def test_InvalidLinePropertyFileTest(self):
- """Test whether an invalid line is ignored."""
- content = '''
- # Foo bar comment.
- default-first-title-mac = blah
-
- # This comment should be ignored.
- crappy-contnet
- foo = bar
- '''
-
- property_file = PropertyFile('test.properties', None, dedent(content))
- expected = {u'default-first-title-mac': u'Foo bar comment.\n',
- u'foo': None}
- parsed = dict([(message.msgid_singular, message.source_comment)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_MultilinePropertyFileTest(self):
- """Test parsing of multiline entries."""
- content = (
- 'multiline-key = This is the first one\\nThis is the second one.')
- property_file = PropertyFile('test.properties', None, content)
- expected = {
- u'multiline-key': (
- [u'This is the first one\nThis is the second one.'])
- }
- parsed = dict([(message.msgid_singular, message.translations)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
- def test_WhiteSpaceBeforeComment(self):
- """Test that single line comment is detected even with white space."""
- content = ' # foo = bar'
- property_file = PropertyFile('test.properties', None, content)
- # No message should be parsed.
- expected = {}
- parsed = dict([(message.msgid_singular, message.translations)
- for message in property_file.messages])
- self.assertEqual(expected, parsed)
-
-
-class MockFile:
- """`TranslationFileData` boiled down to its essence for this test."""
- def __init__(self, path='test.properties', messages=None):
- if messages is None:
- messages = []
- self.path = path
- self.messages = messages
-
-
-class MockMessage:
- """`TranslationMessageData` boiled down to its essence for this test."""
- def __init__(self, msgid, translation, comment=None):
- self.msgid_singular = msgid
- self.translations = [translation]
- self.comment = comment
-
-
-class PropertyFileExportTest(unittest.TestCase):
- """Test XPI `XpiPropertiesSubExporter`."""
-
- def setUp(self):
- self.exporter = XpiPropertiesSubExporter()
-
- def test_properties_export(self):
- # Test plain export of an XPI properties file.
- file = MockFile(messages=[
- MockMessage('foo', 'bar'),
- MockMessage('id', 'translation', comment='comment'),
- ])
-
- expected = dedent("""
- foo=bar
-
- /* comment */
- id=translation
- """).strip()
- self.assertEqual(self.exporter.export(file), expected)
-
- def test_escape(self):
- # Test escaping in properties files.
- file = MockFile(messages=[
- MockMessage("f'oo", 'b"ar', comment="Escaped quotes"),
- MockMessage("f\\oo", "b\\ar", comment="Escaped backslashes"),
- ])
-
- expected = dedent("""
- /* Escaped quotes */
- f\\'oo=b\\"ar
-
- /* Escaped backslashes */
- f\\\\oo=b\\\\ar
- """).strip()
-
- self.assertEqual(self.exporter.export(file).strip(), expected)
-
- def test_escape_comment(self):
- # Test escaping of comments in properties files. Not fancy like
- # actual translation content escaping; just making sure an
- # ill-chosen comment does not produce wildly invalid output.
- file = MockFile(messages=[
- MockMessage("foo", "bar", comment="/*//*/**/ */")])
-
- expected = dedent("""
- /* /*X//*X/**X/ *X/ */
- foo=bar
- """).strip()
-
- self.assertEqual(self.exporter.export(file).strip(), expected)
diff --git a/lib/lp/translations/utilities/tests/test_xpi_search.py b/lib/lp/translations/utilities/tests/test_xpi_search.py
deleted file mode 100644
index 7bd6e74..0000000
--- a/lib/lp/translations/utilities/tests/test_xpi_search.py
+++ /dev/null
@@ -1,83 +0,0 @@
-# Copyright 2009-2017 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Functional tests for searching through XPI POTemplates"""
-__metaclass__ = type
-
-import unittest
-
-from zope.component import getUtility
-
-from lp.registry.interfaces.person import IPersonSet
-from lp.registry.interfaces.product import IProductSet
-from lp.testing.layers import LaunchpadZopelessLayer
-from lp.translations.enums import RosettaImportStatus
-from lp.translations.interfaces.potemplate import IPOTemplateSet
-from lp.translations.utilities.tests.helpers import (
- import_pofile_or_potemplate,
- )
-from lp.translations.utilities.tests.xpi_helpers import (
- get_en_US_xpi_file_to_import,
- )
-
-
-class XpiSearchTestCase(unittest.TestCase):
- """XPI file import into Launchpad."""
-
- layer = LaunchpadZopelessLayer
-
- def setUp(self):
- # Get the importer.
- self.importer = getUtility(IPersonSet).getByName('mark')
-
- # Get the Firefox template.
- firefox_product = getUtility(IProductSet).getByName('firefox')
- firefox_productseries = firefox_product.getSeries('trunk')
- firefox_potemplate_subset = getUtility(IPOTemplateSet).getSubset(
- productseries=firefox_productseries)
- self.firefox_template = firefox_potemplate_subset.new(
- name='firefox',
- translation_domain='firefox',
- path='en-US.xpi',
- owner=self.importer)
- self.spanish_firefox = self.firefox_template.newPOFile('es')
- self.spanish_firefox.path = 'translations/es.xpi'
-
- def setUpTranslationImportQueueForTemplate(self, subdir):
- """Return an ITranslationImportQueueEntry for testing purposes.
-
- :param subdir: subdirectory in firefox-data to get XPI data from.
- """
- # Get the file to import.
- en_US_xpi = get_en_US_xpi_file_to_import(subdir)
- return import_pofile_or_potemplate(
- file_contents=en_US_xpi,
- person=self.importer,
- potemplate=self.firefox_template)
-
- def test_templateSearching(self):
- """Searching through XPI template returns English 'translations'."""
- entry = self.setUpTranslationImportQueueForTemplate('en-US')
-
- # The status is now IMPORTED:
- self.assertEqual(entry.status, RosettaImportStatus.IMPORTED)
-
- potmsgsets = self.spanish_firefox.findPOTMsgSetsContaining(
- text='zilla')
- message_list = [message.singular_text for message in potmsgsets]
-
- self.assertEqual([u'SomeZilla', u'FooZilla!',
- u'FooZilla Zilla Thingy'],
- message_list)
-
- def test_templateSearchingForMsgIDs(self):
- """Searching returns no results for internal msg IDs."""
- entry = self.setUpTranslationImportQueueForTemplate('en-US')
-
- # The status is now IMPORTED:
- self.assertEqual(entry.status, RosettaImportStatus.IMPORTED)
-
- potmsgsets = list(self.spanish_firefox.findPOTMsgSetsContaining(
- text='foozilla.title'))
-
- self.assertEqual(potmsgsets, [])
diff --git a/lib/lp/translations/utilities/tests/xpi_helpers.py b/lib/lp/translations/utilities/tests/xpi_helpers.py
deleted file mode 100644
index 338c343..0000000
--- a/lib/lp/translations/utilities/tests/xpi_helpers.py
+++ /dev/null
@@ -1,82 +0,0 @@
-# Copyright 2009-2018 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-"""Helper methods for XPI testing"""
-__metaclass__ = type
-
-__all__ = [
- 'access_key_source_comment',
- 'command_key_source_comment',
- 'get_en_US_xpi_file_to_import',
- ]
-
-import os.path
-import tempfile
-from textwrap import dedent
-import zipfile
-
-import scandir
-
-import lp.translations
-
-
-command_key_source_comment = dedent(u"""
- Select the shortcut key that you want to use. It should be translated,
- but often shortcut keys (for example Ctrl + KEY) are not changed from
- the original. If a translation already exists, please don't change it
- if you are not sure about it. Please find the context of the key from
- the end of the 'Located in' text below.
- """).strip()
-
-access_key_source_comment = dedent(u"""
- Select the access key that you want to use. These have to be
- translated in a way that the selected character is present in the
- translated string of the label being referred to, for example 'i' in
- 'Edit' menu item in English. If a translation already exists, please
- don't change it if you are not sure about it. Please find the context
- of the key from the end of the 'Located in' text below.
- """).strip()
-
-
-def get_en_US_xpi_file_to_import(subdir):
- """Return an en-US.xpi file object ready to be imported.
-
- The file is generated from utilities/tests/firefox-data/<subdir>.
- """
- # en-US.xpi file is a ZIP file which contains embedded JAR file (which is
- # also a ZIP file) and a couple of other files. Embedded JAR file is
- # named 'en-US.jar' and contains translatable resources.
-
- # Get the root path where the data to generate .xpi file is stored.
- test_root = os.path.join(
- os.path.dirname(lp.translations.__file__),
- 'utilities/tests/firefox-data', subdir)
-
- # First create a en-US.jar file to be included in XPI file.
- jarfile = tempfile.TemporaryFile()
- jar = zipfile.ZipFile(jarfile, 'w')
- jarlist = []
- data_dir = os.path.join(test_root, 'en-US-jar/')
- for root, dirs, files in scandir.walk(data_dir):
- for name in files:
- relative_dir = root[len(data_dir):].strip('/')
- jarlist.append(os.path.join(relative_dir, name))
- for file_name in jarlist:
- f = open(os.path.join(data_dir, file_name), 'r')
- jar.writestr(file_name, f.read())
- jar.close()
- jarfile.seek(0)
-
- # Add remaining bits and en-US.jar to en-US.xpi.
-
- xpifile = tempfile.TemporaryFile()
- xpi = zipfile.ZipFile(xpifile, 'w')
- for xpi_entry in scandir.scandir(test_root):
- if xpi_entry.name != 'en-US-jar':
- with open(xpi_entry.path) as f:
- xpi.writestr(xpi_entry.name, f.read())
- xpi.writestr('chrome/en-US.jar', jarfile.read())
- xpi.close()
- xpifile.seek(0)
-
- return xpifile
diff --git a/lib/lp/translations/utilities/translation_import.py b/lib/lp/translations/utilities/translation_import.py
index c08e5cd..197153d 100644
--- a/lib/lp/translations/utilities/translation_import.py
+++ b/lib/lp/translations/utilities/translation_import.py
@@ -57,7 +57,6 @@ from lp.translations.interfaces.translationmessage import (
from lp.translations.interfaces.translations import TranslationConstants
from lp.translations.utilities.gettext_po_importer import GettextPOImporter
from lp.translations.utilities.kde_po_importer import KdePOImporter
-from lp.translations.utilities.mozilla_xpi_importer import MozillaXpiImporter
from lp.translations.utilities.sanitize import (
sanitize_translations_from_import,
)
@@ -73,7 +72,6 @@ from lp.translations.utilities.validate import (
importers = {
TranslationFileFormat.KDEPO: KdePOImporter(),
TranslationFileFormat.PO: GettextPOImporter(),
- TranslationFileFormat.XPI: MozillaXpiImporter(),
}
diff --git a/lib/lp/translations/utilities/xpi_manifest.py b/lib/lp/translations/utilities/xpi_manifest.py
deleted file mode 100644
index 1bbaedd..0000000
--- a/lib/lp/translations/utilities/xpi_manifest.py
+++ /dev/null
@@ -1,237 +0,0 @@
-# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-__metaclass__ = type
-
-__all__ = ['make_jarpath', 'XpiManifest']
-
-
-import logging
-import re
-
-from lp.translations.interfaces.translationimporter import (
- TranslationFormatSyntaxError,
- )
-
-
-def normalize_path(path):
- """Normalize filesystem path within XPI file."""
- # Normalize "jar:" prefix. Make sure it's there when needed, not there
- # when not needed.
- if path.startswith('jar:'):
- # No leading slashes please.
- path = re.sub('^jar:/+', 'jar:', path)
-
- if '.jar!' not in path:
- logging.debug("Removing 'jar:' from manifest path: '%s'" % path)
- path = path[4:]
- else:
- # No leading slashes please.
- path = re.sub('^/+', '', path)
-
- if '.jar!' in path:
- # Path delves into a jar file, but lacks "jar:" prefix. This is
- # really a malformed path.
- logging.info("Adding 'jar:' to manifest path: '%s'" % path)
- path = 'jar:' + path
-
- # A path inside a jar file must begin with a slash.
- path = path.replace('.jar!', '.jar!/')
-
- # Finally, eliminate redundant slashes. The previous steps may have
- # introduced some.
- return re.sub('/+', '/', path)
-
-
-def is_valid_path(path):
- """Check that path is a valid, normalized path inside an XPI file."""
- if '//' in path:
- return False
- if re.search('\\.jar![^/]', path):
- return False
- if path.startswith('jar:'):
- if path.startswith('jar:jar:'):
- return False
- if '.jar!' not in path:
- return False
- else:
- if '.jar!' in path:
- return False
- return True
-
-
-def is_valid_dir_path(path):
- """Check that path is a normalized directory path in an XPI file."""
- if not is_valid_path(path):
- return False
- if not path.endswith('/'):
- return False
- return True
-
-
-def make_jarpath(path, jarname):
- """Construct base path for files inside a jar file.
-
- To name some translation file that's inside a jar file inside an XPI
- file, concatenate the result of this method (for the jar file) and the
- translation file's path within the jar file.
-
- For example, let's say the XPI file contains foo/bar.jar. Inside
- foo/bar.jar is a translation file locale/gui.dtd. Then
- make_jarfile('foo', 'bar.jar') will return "jar:foo/bar.jar!/", to
- which you can append "locale/gui.dtd" to get the full path
- "jar:foo/bar.jar!/locale/gui.dtd" which identifies the translation
- file within the XPI file.
- """
- # This function is where we drill down into a jar file, so prefix with
- # "jar:" (unless it's already there). We carry the "jar:" prefix only
- # for paths that drill into jar files.
- if not path.startswith('jar:'):
- path = 'jar:' + path
-
- return normalize_path("%s/%s!" % (path, jarname))
-
-
-class ManifestEntry:
- """A "locale" line in a manifest file."""
-
- chrome = None
- locale = None
- path = None
-
- def __init__(self, chrome, locale, path):
- self.chrome = chrome
- self.locale = locale
-
- # Normalize path so we can do simple, reliable text matching on it.
- # The directory paths in an XPI file should end in a single slash.
- # Append the slash here; the normalization will take care of redundant
- # slashes.
- self.path = normalize_path(path + "/")
-
- assert is_valid_dir_path(self.path), (
- "Normalized path not valid: '%s' -> '%s'" % (path, self.path))
-
-
-def manifest_entry_sort_key(entry):
- """We keep manifest entries sorted by path length."""
- return len(entry.path)
-
-
-class XpiManifest:
- """Representation of an XPI manifest file.
-
- Does two things: parsers an XPI file; and looks up chrome paths and
- locales for given filesystem paths inside the XPI file.
- """
-
- # List of locale entries, sorted by increasing path length. The sort
- # order matters for lookup.
- _locales = None
-
- def __init__(self, content):
- """Initialize: parse `content` as a manifest file."""
- if content.startswith('\n'):
- raise TranslationFormatSyntaxError(
- message="Manifest begins with newline.")
-
- locales = []
- for line in content.splitlines():
- words = line.split()
- num_words = len(words)
- if num_words == 0 or words[0] != 'locale':
- pass
- elif num_words < 4:
- logging.info("Ignoring short manifest line: '%s'" % line)
- elif num_words > 4:
- logging.info("Ignoring long manifest line: '%s'" % line)
- else:
- locales.append(ManifestEntry(words[1], words[2], words[3]))
-
- # Eliminate duplicates.
- paths = set()
- deletions = []
- for index, entry in enumerate(locales):
- assert entry.path.endswith('/'), "Manifest path lost its slash"
-
- if entry.path in paths:
- logging.info("Duplicate paths in manifest: '%s'" % entry.path)
- deletions.append(index)
-
- paths.add(entry.path)
-
- for index in reversed(deletions):
- del locales[index]
-
- self._locales = sorted(locales, key=manifest_entry_sort_key)
-
- @classmethod
- def _normalizePath(cls, path):
- """Normalize path. Here so it can be tested without exporting it."""
- return normalize_path(path)
-
- def _getMatchingEntry(self, file_path):
- """Return longest matching entry matching file_path."""
- assert is_valid_path(file_path), (
- "Generated path not valid: %s" % file_path)
-
- # Locale entries are sorted by path length. If we scan backwards, the
- # first entry whose path is a prefix of file_path is the longest
- # match. The fact that the entries' paths have trailing slashes
- # guarantees that we won't match in the middle of a file or directory
- # name.
- for entry in reversed(self._locales):
- if file_path.startswith(entry.path):
- return entry
-
- # No match found.
- return None
-
- def getChromePathAndLocale(self, file_path):
- """Return chrome path and locale applying to a filesystem path.
- """
- assert file_path is not None, "Looking up chrome path for None"
- file_path = self._normalizePath(file_path)
- entry = self._getMatchingEntry(file_path)
-
- if entry is None:
- return None, None
-
- assert file_path.startswith(entry.path), "Found non-matching entry"
- replace = len(entry.path)
- chrome_path = "%s/%s" % (entry.chrome, file_path[replace:])
- return chrome_path, entry.locale
-
- def containsLocales(self, file_path):
- """Is `file_path` a prefix of any path containing locale files?
-
- :param file_path: path of a directory or jar file inside this XPI.
- :return: Boolean: does `file_path` contain locale files?
- """
- file_path = self._normalizePath(file_path)
- for entry in self._locales:
- if entry.path.startswith(file_path):
- return True
- return False
-
- def findMatchingXpiPath(self, chrome_path, locale):
- """Reverse-map a chrome path in a given locale to a file path.
-
- For example, if given "browser/gui/print.dtd" for locale en-US,
- may return "jar:locales/en-US.jar!/chrome/gui/print.dtd",
- assuming that the file path jar:locales/en-US.jar!/chrome/
- is associated with the chrome path browser.
-
- If there are multiple matches, this returns the one with the
- longest file path.
- """
- # Since _locales is sorted by path length, scanning it backwards
- # finds the longest match first.
- for entry in reversed(self._locales):
- is_match = (chrome_path.startswith(entry.chrome + '/') and
- entry.locale == locale)
- if is_match:
- return normalize_path(
- entry.path + chrome_path[len(entry.chrome):])
-
- return None
diff --git a/lib/lp/translations/utilities/xpi_properties_exporter.py b/lib/lp/translations/utilities/xpi_properties_exporter.py
deleted file mode 100644
index 43cf397..0000000
--- a/lib/lp/translations/utilities/xpi_properties_exporter.py
+++ /dev/null
@@ -1,53 +0,0 @@
-# Copyright 2009 Canonical Ltd. This software is licensed under the
-# GNU Affero General Public License version 3 (see the file LICENSE).
-
-__metaclass__ = type
-
-__all__ = [
- 'XpiPropertiesSubExporter'
- ]
-
-
-import re
-
-
-def has_comment(message):
- """Does `TranslationMessageData` contain a comment?"""
- return message.comment is not None and message.comment.strip() != ''
-
-
-class XpiPropertiesSubExporter:
- """Produce a properties file to go into an XPI file."""
-
- def _escape(self, string):
- """Escape message string for use in properties file."""
- # Escape backslashes first, before we start inserting ones of
- # our own. Then the other stuff. Replace newlines by \n etc.,
- # and encode non-ASCII characters as \uXXXX.
- string = string.replace('\\', r'\\')
- string = re.sub('''(["'])''', r'\\\1', string)
- # Escape newlines as \n etc, and non-ASCII as \uXXXX
- return string.encode('ascii', 'backslashreplace')
-
- def _escape_comment(self, comment):
- """Escape comment string for use in properties file."""
- # Prevent comment from breaking out of /* ... */ block.
- comment = comment.replace('*/', '*X/')
- return comment.encode('ascii', 'unicode-escape')
-
- def export(self, translation_file):
- assert translation_file.path.endswith('.properties'), (
- "Unexpected properties file suffix: %s" % translation_file.path)
- contents = []
- for message in translation_file.messages:
- if not message.translations:
- continue
- if has_comment(message):
- contents.append(
- "\n/* %s */" % self._escape_comment(message.comment))
- msgid = self._escape(message.msgid_singular)
- text = self._escape(message.translations[0])
- line = "%s=%s" % (msgid, text)
- contents.append(line)
-
- return '\n'.join(contents)
diff --git a/utilities/sourcedeps.cache b/utilities/sourcedeps.cache
index 412ea52..fef3955 100644
--- a/utilities/sourcedeps.cache
+++ b/utilities/sourcedeps.cache
@@ -27,10 +27,6 @@
494,
"cjwatson@xxxxxxxxxxxxx-20190919081036-q1symc2h2iedtlh3"
],
- "old_xmlplus": [
- 4,
- "sinzui-20090526164636-1swugzupwvjgomo4"
- ],
"pygettextpo": [
25,
"launchpad@xxxxxxxxxxxxxxxxx-20140116030912-lqm1dtb6a0y4femq"
diff --git a/utilities/sourcedeps.conf b/utilities/sourcedeps.conf
index b19007d..13287d4 100644
--- a/utilities/sourcedeps.conf
+++ b/utilities/sourcedeps.conf
@@ -14,5 +14,4 @@ bzr-svn lp:~launchpad-pqm/bzr-svn/devel;revno=2725
cscvs lp:~launchpad-pqm/launchpad-cscvs/devel;revno=433
difftacular lp:~launchpad/difftacular/trunk;revno=11
loggerhead lp:~loggerhead-team/loggerhead/trunk-rich;revno=494
-old_xmlplus lp:~launchpad-pqm/dtdparser/trunk;revno=4
pygettextpo lp:~launchpad-pqm/pygettextpo/trunk;revno=25
