launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #04444
[Merge] lp:~danilo/launchpad/bug-814580 into lp:launchpad
Данило Шеган has proposed merging lp:~danilo/launchpad/bug-814580 into lp:launchpad with lp:~danilo/launchpad/bug-814580-db as a prerequisite.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers)
For more details, see:
https://code.launchpad.net/~danilo/launchpad/bug-814580/+merge/69978
= Bug 814580: migrate translations on POTemplate changes =
== Proposed fix ==
This extends the TranslationSharingJob to do translation splitting/merging even when a single template is modified (i.e. renamed or moved to a different parent).
We use the existing code for translation merging and only add new code for finding which templates are affected, and do the similar thing for translation splitting.
I haven't fixed the lint yet so as not to make the diff larger. I'll happily do that before landing.
== Tests ==
bin/test -cvvt translationsplitter -t translationpackaging
== Demo and Q/A ==
Rename a PO template, note how TranslationSharingJob is created (table PackagingJob, in the process of being renamed).
Test how the job works by executing cronscripts/run_jobs.py -vv packaging_translations
(also move the po template to a different project/sourcepackage and confirm the same happens)
Use the POTemplate:+admin page to rename it or move it to a different project/source package.
= Launchpad lint =
Checking for conflicts and issues in changed files.
Linting changed files:
lib/lp/translations/utilities/translationsplitter.py
lib/lp/translations/interfaces/translationsharingjob.py
lib/lp/translations/translationmerger.py
lib/lp/translations/model/translationsharingjob.py
lib/lp/translations/model/translationpackagingjob.py
lib/lp/translations/tests/test_translationpackagingjob.py
lib/lp/translations/configure.zcml
lib/lp/translations/tests/test_translationsplitter.py
./lib/lp/translations/translationmerger.py
571: local variable 'total_ids' is assigned to but never used
./lib/lp/translations/tests/test_translationpackagingjob.py
63: local variable 'package' is assigned to but never used
196: local variable 'recorder' is assigned to but never used
207: local variable 'recorder' is assigned to but never used
215: local variable 'other_packaging' is assigned to but never used
229: local variable 'job' is assigned to but never used
227: local variable 'recorder' is assigned to but never used
236: local variable 'recorder' is assigned to but never used
253: local variable 'recorder' is assigned to but never used
257: local variable 'job2' is assigned to but never used
./lib/lp/translations/tests/test_translationsplitter.py
81: local variable 'ubuntu_template' is assigned to but never used
82: local variable 'ubuntu_sequence' is assigned to but never used
95: local variable 'upstream_translation' is assigned to but never used
112: local variable 'upstream_translation' is assigned to but never used
143: local variable 'upstream_message' is assigned to but never used
--
https://code.launchpad.net/~danilo/launchpad/bug-814580/+merge/69978
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~danilo/launchpad/bug-814580 into lp:launchpad.
=== modified file 'lib/lp/translations/configure.zcml'
--- lib/lp/translations/configure.zcml 2011-08-01 10:36:21 +0000
+++ lib/lp/translations/configure.zcml 2011-08-01 10:36:24 +0000
@@ -154,6 +154,10 @@
for="lp.registry.interfaces.packaging.IPackaging
lazr.lifecycle.interfaces.IObjectEvent"
handler=".model.translationsharingjob.schedule_packaging_job" />
+ <subscriber
+ for="lp.translations.interfaces.potemplate.IPOTemplate
+ lazr.lifecycle.interfaces.IObjectModifiedEvent"
+ handler=".model.translationsharingjob.schedule_potemplate_job" />
<facet
facet="translations">
@@ -643,6 +647,10 @@
class="lp.translations.model.translationpackagingjob.TranslationSplitJob">
<allow interface='lp.services.job.interfaces.job.IRunnableJob'/>
</class>
+ <class
+ class="lp.translations.model.translationpackagingjob.TranslationTemplateChangeJob">
+ <allow interface='lp.services.job.interfaces.job.IRunnableJob'/>
+ </class>
<utility
component="lp.translations.model.translationtemplatesbuildjob.TranslationTemplatesBuildJob"
provides="lp.buildmaster.interfaces.buildfarmjob.IBuildFarmJob"
=== modified file 'lib/lp/translations/interfaces/translationsharingjob.py'
--- lib/lp/translations/interfaces/translationsharingjob.py 2011-08-01 10:36:21 +0000
+++ lib/lp/translations/interfaces/translationsharingjob.py 2011-08-01 10:36:24 +0000
@@ -19,3 +19,6 @@
sourcepackagename = Attribute(
_("The sourcepackagename of the Packaging."))
+
+ potemplate = Attribute(
+ _("The POTemplate to pass around as the relevant template."))
=== modified file 'lib/lp/translations/model/translationpackagingjob.py'
--- lib/lp/translations/model/translationpackagingjob.py 2011-08-01 10:36:21 +0000
+++ lib/lp/translations/model/translationpackagingjob.py 2011-08-01 10:36:24 +0000
@@ -10,6 +10,7 @@
__all__ = [
'TranslationMergeJob',
'TranslationSplitJob',
+ 'TranslationTemplateChangeJob',
]
import logging
@@ -17,6 +18,7 @@
from lazr.lifecycle.interfaces import (
IObjectCreatedEvent,
IObjectDeletedEvent,
+ IObjectModifiedEvent,
)
import transaction
from zope.interface import (
@@ -40,7 +42,10 @@
TransactionManager,
TranslationMerger,
)
-from lp.translations.utilities.translationsplitter import TranslationSplitter
+from lp.translations.utilities.translationsplitter import (
+ TranslationSplitter,
+ TranslationTemplateSplitter,
+ )
class TranslationPackagingJob(TranslationSharingJobDerived, BaseRunnableJob):
@@ -117,3 +122,31 @@
'Splitting %s and %s', self.productseries.displayname,
self.sourcepackage.displayname)
TranslationSplitter(self.productseries, self.sourcepackage).split()
+
+
+class TranslationTemplateChangeJob(TranslationPackagingJob):
+ """Job for merging/splitting translations when template is changed."""
+
+ implements(IRunnableJob)
+
+ class_job_type = TranslationSharingJobType.TEMPLATE_CHANGE
+
+ create_on_event = IObjectModifiedEvent
+
+ @classmethod
+ def forPOTemplate(cls, potemplate):
+ """Create a TranslationTemplateChangeJob for a POTemplate.
+
+ :param potemplate: The `POTemplate` to create the job for.
+ :return: A `TranslationTemplateChangeJob`.
+ """
+ return cls.create(potemplate=potemplate)
+
+ def run(self):
+ """See `IRunnableJob`."""
+ logger = logging.getLogger()
+ logger.info("Sanitizing translations for '%s'" % (
+ self.potemplate.displayname))
+ TranslationTemplateSplitter(self.potemplate).split()
+ tm = TransactionManager(transaction.manager, False)
+ TranslationMerger.mergeModifiedTemplates(self.potemplate, tm)
=== modified file 'lib/lp/translations/model/translationsharingjob.py'
--- lib/lp/translations/model/translationsharingjob.py 2011-08-01 10:36:21 +0000
+++ lib/lp/translations/model/translationsharingjob.py 2011-08-01 10:36:24 +0000
@@ -37,6 +37,7 @@
from lp.translations.interfaces.translationsharingjob import (
ITranslationSharingJob,
)
+from lp.translations.model.potemplate import POTemplate
class TranslationSharingJobType(DBEnumeratedType):
@@ -54,6 +55,12 @@
Split translations between productseries and sourcepackage.
""")
+ TEMPLATE_CHANGE = DBItem(2, """
+ Split/merge translations for a single translation template.
+
+ Split/merge translations for a single translation template.
+ """)
+
class TranslationSharingJob(StormBase):
"""Base class for jobs related to a packaging."""
@@ -82,8 +89,12 @@
sourcepackagename = Reference(sourcepackagename_id, SourcePackageName.id)
+ potemplate_id = Int('potemplate')
+
+ potemplate = Reference(potemplate_id, POTemplate.id)
+
def __init__(self, job, job_type, productseries, distroseries,
- sourcepackagename):
+ sourcepackagename, potemplate=None):
""""Constructor.
:param job: The `Job` to use for storing basic job state.
@@ -96,6 +107,7 @@
self.distroseries = distroseries
self.sourcepackagename = sourcepackagename
self.productseries = productseries
+ self.potemplate = potemplate
class RegisteredSubclass(type):
@@ -143,16 +155,18 @@
self.job = job
@classmethod
- def create(cls, productseries, distroseries, sourcepackagename):
+ def create(cls, productseries=None, distroseries=None,
+ sourcepackagename=None, potemplate=None):
""""Create a TranslationPackagingJob backed by TranslationSharingJob.
:param productseries: The ProductSeries side of the Packaging.
:param distroseries: The distroseries of the Packaging sourcepackage.
:param sourcepackagename: The name of the Packaging sourcepackage.
+ :param potemplate: POTemplate to restrict to (if any).
"""
context = TranslationSharingJob(
Job(), cls.class_job_type, productseries,
- distroseries, sourcepackagename)
+ distroseries, sourcepackagename, potemplate)
return cls(context)
@classmethod
@@ -170,6 +184,27 @@
job_class.forPackaging(packaging)
@classmethod
+ def schedulePOTemplateJob(cls, potemplate, event):
+ """Event subscriber to create a TranslationSharingJob on events.
+
+ :param potemplate: The `POTemplate` to create
+ a `TranslationSharingJob` for.
+ :param event: The event itself.
+ """
+ if ('name' not in event.edited_fields and
+ 'productseries' not in event.edited_fields and
+ 'distroseries' not in event.edited_fields and
+ 'sourcepackagename' not in event.edited_fields):
+ # Ignore changes to POTemplates that are neither renames,
+ # nor moves to a different package/project.
+ return
+ for event_type, job_classes in cls._event_types.iteritems():
+ if not event_type.providedBy(event):
+ continue
+ for job_class in job_classes:
+ job_class.forPOTemplate(potemplate)
+
+ @classmethod
def iterReady(cls, extra_clauses):
"""See `IJobSource`.
@@ -207,3 +242,4 @@
#make accessible to zcml
schedule_packaging_job = TranslationSharingJobDerived.schedulePackagingJob
+schedule_potemplate_job = TranslationSharingJobDerived.schedulePOTemplateJob
=== modified file 'lib/lp/translations/tests/test_translationpackagingjob.py'
--- lib/lp/translations/tests/test_translationpackagingjob.py 2011-08-01 10:36:21 +0000
+++ lib/lp/translations/tests/test_translationpackagingjob.py 2011-08-01 10:36:24 +0000
@@ -36,6 +36,7 @@
TranslationMergeJob,
TranslationPackagingJob,
TranslationSplitJob,
+ TranslationTemplateChangeJob,
)
from lp.translations.tests.test_translationsplitter import (
make_shared_potmsgset,
@@ -101,20 +102,31 @@
class JobFinder:
- def __init__(self, productseries, sourcepackage, job_class):
- self.productseries = productseries
- self.sourcepackagename = sourcepackage.sourcepackagename
- self.distroseries = sourcepackage.distroseries
+ def __init__(self, productseries, sourcepackage, job_class,
+ potemplate=None):
+ if potemplate is None:
+ self.productseries = productseries
+ self.sourcepackagename = sourcepackage.sourcepackagename
+ self.distroseries = sourcepackage.distroseries
+ else:
+ self.potemplate = potemplate
self.job_type = job_class.class_job_type
def find(self):
- return list(TranslationSharingJobDerived.iterReady([
- TranslationSharingJob.productseries_id == self.productseries.id,
- (TranslationSharingJob.sourcepackagename_id ==
- self.sourcepackagename.id),
- TranslationSharingJob.distroseries_id == self.distroseries.id,
- TranslationSharingJob.job_type == self.job_type,
- ]))
+ if self.potemplate is None:
+ return list(TranslationSharingJobDerived.iterReady([
+ TranslationSharingJob.productseries_id == self.productseries.id,
+ (TranslationSharingJob.sourcepackagename_id ==
+ self.sourcepackagename.id),
+ TranslationSharingJob.distroseries_id == self.distroseries.id,
+ TranslationSharingJob.job_type == self.job_type,
+ ]))
+ else:
+ return list(
+ TranslationSharingJobDerived.iterReady([
+ TranslationSharingJob.potemplate_id == self.potemplate.id,
+ TranslationSharingJob.job_type == self.job_type,
+ ]))
class TestTranslationPackagingJob(TestCaseWithFactory):
@@ -275,3 +287,59 @@
packaging.distroseries)
(job,) = finder.find()
self.assertIsInstance(job, TranslationSplitJob)
+
+
+class TestTranslationTemplateChangeJob(TestCaseWithFactory):
+
+ layer = LaunchpadZopelessLayer
+
+ def test_modifyPOTemplate_makes_job(self):
+ """Creating a Packaging should make a TranslationMergeJob."""
+ potemplate = self.factory.makePOTemplate()
+ finder = JobFinder(
+ None, None, TranslationTemplateChangeJob, potemplate)
+ self.assertEqual([], finder.find())
+ with person_logged_in(potemplate.owner):
+ potemplate.name = self.factory.getUniqueString()
+ (job,) = finder.find()
+ self.assertIsInstance(job, TranslationTemplateChangeJob)
+
+ def test_splits_and_merges(self):
+ """Changing a template makes the translations split and then
+ re-merged in the new target sharing set."""
+ potemplate = self.factory.makePOTemplate(name='template')
+ other_ps = self.factory.makeProductSeries(
+ product=potemplate.productseries.product)
+ old_shared = self.factory.makePOTemplate(name='template',
+ productseries=other_ps)
+ new_shared = self.factory.makePOTemplate(name='renamed',
+ productseries=other_ps)
+
+ # Set up shared POTMsgSets and translations.
+ potmsgset = self.factory.makePOTMsgSet(potemplate, sequence=1)
+ potmsgset.setSequence(old_shared, 1)
+ self.factory.makeCurrentTranslationMessage(potmsgset=potmsgset)
+
+ # This is the identical English message in the new_shared template.
+ target_potmsgset = self.factory.makePOTMsgSet(
+ new_shared, sequence=1, singular=potmsgset.singular_text)
+
+ # Rename the template and confirm that messages are now shared
+ # with new_shared instead of old_shared.
+ potemplate.name = 'renamed'
+ job = TranslationTemplateChangeJob.create(potemplate=potemplate)
+
+ self.becomeDbUser('rosettaadmin')
+ job.run()
+
+ # New POTMsgSet is now different from the old one (it's been split),
+ # but matches the target potmsgset (it's been merged into it).
+ new_potmsgset = potemplate.getPOTMsgSets()[0]
+ self.assertNotEqual(potmsgset, new_potmsgset)
+ self.assertEqual(target_potmsgset, new_potmsgset)
+
+ # Translations have been merged as well.
+ self.assertContentEqual(
+ [tm.translations for tm in potmsgset.getAllTranslationMessages()],
+ [tm.translations
+ for tm in new_potmsgset.getAllTranslationMessages()])
=== modified file 'lib/lp/translations/tests/test_translationsplitter.py'
--- lib/lp/translations/tests/test_translationsplitter.py 2011-02-25 20:23:40 +0000
+++ lib/lp/translations/tests/test_translationsplitter.py 2011-08-01 10:36:24 +0000
@@ -13,6 +13,7 @@
)
from lp.translations.utilities.translationsplitter import (
TranslationSplitter,
+ TranslationTemplateSplitter,
)
@@ -153,3 +154,183 @@
upstream_item.potmsgset.getAllTranslationMessages().count(),
ubuntu_item.potmsgset.getAllTranslationMessages().count(),
)
+
+
+class TestTranslationTemplateSplitterBase:
+
+ layer = ZopelessDatabaseLayer
+
+ def getPOTMsgSetAndTemplateToSplit(self, splitter):
+ return [(tti1.potmsgset, tti1.potemplate)
+ for tti1, tti2 in splitter.findShared()]
+
+ def setUpSharingTemplates(self, other_side=False):
+ """Sets up two sharing templates with one sharing message and
+ one non-sharing message in each template."""
+ template1 = self.makePOTemplate()
+ template2 = self.makeSharingTemplate(template1, other_side)
+
+ shared_potmsgset = self.factory.makePOTMsgSet(template1, sequence=1)
+ shared_potmsgset.setSequence(template2, 1)
+
+ # POTMsgSets appearing in only one of the templates are not returned.
+ self.factory.makePOTMsgSet(template1, sequence=2)
+ self.factory.makePOTMsgSet(template2, sequence=2)
+ return template1, template2, shared_potmsgset
+
+ def makePOTemplate(self):
+ raise NotImplementedError('Subclasses should implement this.')
+
+ def makeSharingTemplate(self, template, other_side=False):
+ raise NotImplementedError('Subclasses should implement this.')
+
+ def test_findShared_renamed(self):
+ """Shared POTMsgSets are included for a renamed template."""
+ template1, template2, shared_potmsgset = self.setUpSharingTemplates()
+
+ splitter = TranslationTemplateSplitter(template2)
+ self.assertContentEqual([], splitter.findShared())
+
+ template2.name = 'renamed'
+ self.assertContentEqual(
+ [(shared_potmsgset, template1)],
+ self.getPOTMsgSetAndTemplateToSplit(splitter))
+
+ def test_findShared_moved_product(self):
+ """Moving a template to a different product splits its messages."""
+ template1, template2, shared_potmsgset = self.setUpSharingTemplates()
+
+ splitter = TranslationTemplateSplitter(template2)
+ self.assertContentEqual([], splitter.findShared())
+
+ # Move the template to a different product entirely.
+ template2.productseries = self.factory.makeProduct().development_focus
+ template2.distroseries = None
+ template2.sourcepackagename = None
+ self.assertContentEqual(
+ [(shared_potmsgset, template1)],
+ self.getPOTMsgSetAndTemplateToSplit(splitter))
+
+ def test_findShared_moved_distribution(self):
+ """Moving a template to a different distribution gets it split."""
+ template1, template2, shared_potmsgset = self.setUpSharingTemplates()
+
+ splitter = TranslationTemplateSplitter(template2)
+ self.assertContentEqual([], splitter.findShared())
+
+ # Move the template to a different distribution entirely.
+ sourcepackage = self.factory.makeSourcePackage()
+ template2.distroseries = sourcepackage.distroseries
+ template2.sourcepackagename = sourcepackage.sourcepackagename
+ template2.productseries = None
+ self.assertContentEqual(
+ [(shared_potmsgset, template1)],
+ self.getPOTMsgSetAndTemplateToSplit(splitter))
+
+ def test_findShared_moved_to_nonsharing_target(self):
+ """Moving a template to a target not sharing with the existing
+ upstreams and source package gets it split."""
+ template1, template2, shared_potmsgset = self.setUpSharingTemplates(
+ other_side=True)
+
+ splitter = TranslationTemplateSplitter(template2)
+ self.assertContentEqual([], splitter.findShared())
+
+ # Move the template to a different distribution entirely.
+ sourcepackage = self.factory.makeSourcePackage()
+ template2.distroseries = sourcepackage.distroseries
+ template2.sourcepackagename = sourcepackage.sourcepackagename
+ template2.productseries = None
+ self.assertContentEqual(
+ [(shared_potmsgset, template1)],
+ self.getPOTMsgSetAndTemplateToSplit(splitter))
+
+ def test_split_messages(self):
+ """Splitting messages works properly."""
+ template1, template2, shared_potmsgset = self.setUpSharingTemplates()
+
+ splitter = TranslationTemplateSplitter(template2)
+ self.assertContentEqual([], splitter.findShared())
+
+ # Move the template to a different product entirely.
+ template2.productseries = self.factory.makeProduct().development_focus
+ template2.distroseries = None
+ template2.sourcepackagename = None
+
+ other_item, this_item = splitter.findShared()[0]
+
+ splitter.split()
+
+ self.assertNotEqual(other_item.potmsgset, this_item.potmsgset)
+ self.assertEqual(shared_potmsgset, other_item.potmsgset)
+ self.assertNotEqual(shared_potmsgset, this_item.potmsgset)
+
+
+class TestProductTranslationTemplateSplitter(
+ TestCaseWithFactory, TestTranslationTemplateSplitterBase):
+ """Templates in a product get split appropriately."""
+
+ def makePOTemplate(self):
+ return self.factory.makePOTemplate(
+ name='template',
+ side=TranslationSide.UPSTREAM)
+
+ def makeSharingTemplate(self, template, other_side=False):
+ if other_side:
+ template2 = self.factory.makePOTemplate(
+ name='template',
+ side=TranslationSide.UBUNTU)
+ self.factory.makePackagingLink(
+ productseries=template.productseries,
+ distroseries=template2.distroseries,
+ sourcepackagename=template2.sourcepackagename)
+ return template2
+ else:
+ product = template.productseries.product
+ other_series = self.factory.makeProductSeries(product=product)
+ return self.factory.makePOTemplate(name='template',
+ productseries=other_series)
+
+
+class TestDistributionTranslationTemplateSplitter(
+ TestCaseWithFactory, TestTranslationTemplateSplitterBase):
+ """Templates in a distribution get split appropriately."""
+
+ def makePOTemplate(self):
+ return self.factory.makePOTemplate(
+ name='template',
+ side=TranslationSide.UBUNTU)
+
+ def makeSharingTemplate(self, template, other_side=False):
+ if other_side:
+ template2 = self.factory.makePOTemplate(
+ name='template',
+ side=TranslationSide.UPSTREAM)
+ self.factory.makePackagingLink(
+ productseries=template2.productseries,
+ distroseries=template.distroseries,
+ sourcepackagename=template.sourcepackagename)
+ return template2
+ else:
+ distro = template.distroseries.distribution
+ other_series = self.factory.makeDistroRelease(distribution=distro)
+ return self.factory.makePOTemplate(
+ name='template',
+ distroseries=other_series,
+ sourcepackagename=template.sourcepackagename)
+
+ def test_findShared_moved_sourcepackage(self):
+ """Moving a template to a different source package gets it split."""
+ template1, template2, shared_potmsgset = self.setUpSharingTemplates()
+
+ splitter = TranslationTemplateSplitter(template2)
+ self.assertContentEqual([], splitter.findShared())
+
+ # Move the template to a different source package inside the
+ # same distroseries.
+ sourcepackage = self.factory.makeSourcePackage(
+ distroseries=template2.distroseries)
+ template2.sourcepackagename = sourcepackage.sourcepackagename
+ self.assertContentEqual(
+ [(shared_potmsgset, template1)],
+ self.getPOTMsgSetAndTemplateToSplit(splitter))
=== modified file 'lib/lp/translations/translationmerger.py'
--- lib/lp/translations/translationmerger.py 2011-05-27 21:12:25 +0000
+++ lib/lp/translations/translationmerger.py 2011-08-01 10:36:24 +0000
@@ -387,6 +387,26 @@
merger = cls(templates, tm)
merger.mergePOTMsgSets()
+ @classmethod
+ def mergeModifiedTemplates(cls, potemplate, tm):
+ subset = getUtility(IPOTemplateSet).getSharingSubset(
+ distribution=potemplate.distribution,
+ sourcepackagename=potemplate.sourcepackagename,
+ product=potemplate.product)
+ templates = list(subset.getSharingPOTemplates(potemplate.name))
+ templates.sort(key=methodcaller('sharingKey'), reverse=True)
+ merger = cls(templates, tm)
+ merger.mergeAll()
+
+ def mergeAll(self):
+ """Properly merge POTMsgSets and TranslationMessages."""
+ self._removeDuplicateMessages()
+ self.tm.endTransaction(intermediate=True)
+ self.mergePOTMsgSets()
+ self.tm.endTransaction(intermediate=True)
+ self.mergeTranslationMessages()
+ self.tm.endTransaction()
+
def __init__(self, potemplates, tm):
"""Constructor.
=== modified file 'lib/lp/translations/utilities/translationsplitter.py'
--- lib/lp/translations/utilities/translationsplitter.py 2011-05-12 20:21:58 +0000
+++ lib/lp/translations/utilities/translationsplitter.py 2011-08-01 10:36:24 +0000
@@ -6,50 +6,30 @@
import logging
-from storm.locals import ClassAlias, Store
+from storm.expr import (
+ And,
+ Join,
+ LeftJoin,
+ Not,
+ Or,
+ )
+from storm.locals import (
+ ClassAlias,
+ Store,
+ )
import transaction
+from lp.registry.model.distroseries import DistroSeries
+from lp.registry.model.packaging import Packaging
+from lp.registry.model.productseries import ProductSeries
from lp.translations.model.potemplate import POTemplate
from lp.translations.model.translationtemplateitem import (
TranslationTemplateItem,
)
-class TranslationSplitter:
- """Split translations for a productseries, sourcepackage pair.
-
- If a productseries and sourcepackage were linked in error, and then
- unlinked, they may still share some translations. This class breaks those
- associations.
- """
-
- def __init__(self, productseries, sourcepackage):
- """Constructor.
-
- :param productseries: The `ProductSeries` to split from.
- :param sourcepackage: The `SourcePackage` to split from.
- """
- self.productseries = productseries
- self.sourcepackage = sourcepackage
-
- def findShared(self):
- """Provide tuples of upstream, ubuntu for each shared POTMsgSet."""
- store = Store.of(self.productseries)
- UpstreamItem = ClassAlias(TranslationTemplateItem, 'UpstreamItem')
- UpstreamTemplate = ClassAlias(POTemplate, 'UpstreamTemplate')
- UbuntuItem = ClassAlias(TranslationTemplateItem, 'UbuntuItem')
- UbuntuTemplate = ClassAlias(POTemplate, 'UbuntuTemplate')
- return store.find(
- (UpstreamItem, UbuntuItem),
- UpstreamItem.potmsgsetID == UbuntuItem.potmsgsetID,
- UbuntuItem.potemplateID == UbuntuTemplate.id,
- UbuntuTemplate.sourcepackagenameID ==
- self.sourcepackage.sourcepackagename.id,
- UbuntuTemplate.distroseriesID ==
- self.sourcepackage.distroseries.id,
- UpstreamItem.potemplateID == UpstreamTemplate.id,
- UpstreamTemplate.productseriesID == self.productseries.id,
- )
+class TranslationSplitterBase:
+ """Base class for translation splitting jobs."""
@staticmethod
def splitPOTMsgSet(ubuntu_item):
@@ -86,9 +66,151 @@
"""Split the translations for the ProductSeries and SourcePackage."""
logger = logging.getLogger()
shared = enumerate(self.findShared(), 1)
+ total = 0
for num, (upstream_item, ubuntu_item) in shared:
self.splitPOTMsgSet(ubuntu_item)
self.migrateTranslations(upstream_item.potmsgset, ubuntu_item)
if num % 100 == 0:
logger.info('%d entries split. Committing...', num)
transaction.commit()
+ total = num
+
+ if total % 100 != 0 or total == 0:
+ transaction.commit()
+ logger.info('%d entries split.', total)
+
+
+class TranslationSplitter(TranslationSplitterBase):
+ """Split translations for a productseries, sourcepackage pair.
+
+ If a productseries and sourcepackage were linked in error, and then
+ unlinked, they may still share some translations. This class breaks those
+ associations.
+ """
+
+ def __init__(self, productseries, sourcepackage):
+ """Constructor.
+
+ :param productseries: The `ProductSeries` to split from.
+ :param sourcepackage: The `SourcePackage` to split from.
+ """
+ self.productseries = productseries
+ self.sourcepackage = sourcepackage
+
+ def findShared(self):
+ """Provide tuples of upstream, ubuntu for each shared POTMsgSet."""
+ store = Store.of(self.productseries)
+ UpstreamItem = ClassAlias(TranslationTemplateItem, 'UpstreamItem')
+ UpstreamTemplate = ClassAlias(POTemplate, 'UpstreamTemplate')
+ UbuntuItem = ClassAlias(TranslationTemplateItem, 'UbuntuItem')
+ UbuntuTemplate = ClassAlias(POTemplate, 'UbuntuTemplate')
+ return store.find(
+ (UpstreamItem, UbuntuItem),
+ UpstreamItem.potmsgsetID == UbuntuItem.potmsgsetID,
+ UbuntuItem.potemplateID == UbuntuTemplate.id,
+ UbuntuTemplate.sourcepackagenameID ==
+ self.sourcepackage.sourcepackagename.id,
+ UbuntuTemplate.distroseriesID ==
+ self.sourcepackage.distroseries.id,
+ UpstreamItem.potemplateID == UpstreamTemplate.id,
+ UpstreamTemplate.productseriesID == self.productseries.id,
+ )
+
+
+class TranslationTemplateSplitter(TranslationSplitterBase):
+ """Split translations for an extracted potemplate.
+
+ When a POTemplate is removed from a set of sharing templates,
+ it keeps sharing POTMsgSets with other templates. This class
+ removes those associations.
+ """
+
+ def __init__(self, potemplate):
+ """Constructor.
+
+ :param potemplate: The `POTemplate` to sanitize.
+ """
+ self.potemplate = potemplate
+
+ def findShared(self):
+ """Provide tuples of (other, this) items for each shared POTMsgSet.
+
+ Only return those that are shared but shouldn't be because they
+ are now in non-sharing templates.
+ """
+ store = Store.of(self.potemplate)
+ ThisItem = ClassAlias(TranslationTemplateItem, 'ThisItem')
+ OtherItem = ClassAlias(TranslationTemplateItem, 'OtherItem')
+ OtherTemplate = ClassAlias(POTemplate, 'OtherTemplate')
+
+ tables = [
+ OtherTemplate,
+ Join(OtherItem, OtherItem.potemplateID == OtherTemplate.id),
+ Join(ThisItem,
+ And(ThisItem.potmsgsetID == OtherItem.potmsgsetID,
+ ThisItem.potemplateID == self.potemplate.id)),
+ ]
+
+ if self.potemplate.productseries is not None:
+ # If the template is now in a product, we look for all
+ # effectively sharing templates that are in *different*
+ # products, or that are in a sourcepackage which is not
+ # linked (through Packaging table) with this product.
+ ps = self.potemplate.productseries
+ productseries_join = LeftJoin(
+ ProductSeries,
+ ProductSeries.id == OtherTemplate.productseriesID)
+ packaging_join = LeftJoin(
+ Packaging,
+ And(Packaging.productseriesID == ps.id,
+ (Packaging.sourcepackagenameID ==
+ OtherTemplate.sourcepackagenameID),
+ Packaging.distroseriesID == OtherTemplate.distroseriesID
+ ))
+ tables.extend([productseries_join, packaging_join])
+ # Template should not be sharing if...
+ other_clauses = Or(
+ # The name is different, or...
+ OtherTemplate.name != self.potemplate.name,
+ # It's in a different product, or...
+ And(Not(ProductSeries.id == None),
+ ProductSeries.productID != ps.productID),
+ # There is no link between this product series and
+ # a source package the template is in.
+ And(Not(OtherTemplate.distroseriesID == None),
+ Packaging.id == None))
+ else:
+ # If the template is now in a source package, we look for all
+ # effectively sharing templates that are in *different*
+ # distributions or source packages, or that are in a product
+ # which is not linked with this source package.
+ ds = self.potemplate.distroseries
+ spn = self.potemplate.sourcepackagename
+ distroseries_join = LeftJoin(
+ DistroSeries,
+ DistroSeries.id == OtherTemplate.distroseriesID)
+ packaging_join = LeftJoin(
+ Packaging,
+ And(Packaging.distroseriesID == ds.id,
+ Packaging.sourcepackagenameID == spn.id,
+ Packaging.productseriesID == OtherTemplate.productseriesID
+ ))
+ tables.extend([distroseries_join, packaging_join])
+ # Template should not be sharing if...
+ other_clauses = Or(
+ # The name is different, or...
+ OtherTemplate.name != self.potemplate.name,
+ # It's in a different distribution or source package, or...
+ And(Not(DistroSeries.id == None),
+ Or(DistroSeries.distributionID != ds.distributionID,
+ OtherTemplate.sourcepackagenameID != spn.id)),
+ # There is no link between this source package and
+ # a product the template is in.
+ And(Not(OtherTemplate.productseriesID == None),
+ Packaging.id == None))
+
+ return store.using(*tables).find(
+ (OtherItem, ThisItem),
+ OtherTemplate.id != self.potemplate.id,
+ other_clauses,
+ )
