launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #02053
[Merge] lp:~jtv/launchpad/recife-statistics into lp:launchpad/db-devel
Jeroen T. Vermeulen has proposed merging lp:~jtv/launchpad/recife-statistics into lp:launchpad/db-devel with lp:~jtv/launchpad/recife-pre-stats as a prerequisite.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers): code
= "Recife" POFile statistics =
This branch re-does the computation of translation statistics on POFiles for our new "Recife" data model.
It's not a complete branch. The change breaks tests that I will fix in separate branches, and there are more behaviours to test for and support, but I had to cut off for review somewhere. That's also why the BRANCH.TODO contains to-do items. Our landing systems won't accept the branch until I remove those—which I will do as I finish with each of them.
The statistics calculations end up being simpler than they were previously because as it turns out, most of the relevant numbers can be gathered through a single query. This also makes their relationships clearer, including one wart: the "Rosetta count" (which in the new model means new translations in Ubuntu that aren't upstream if you're looking at an Ubuntu translation, or new translations in the upstream project that aren't in Ubuntu if you're looking at an upstream translation) includes both the "updates count" (messages that are translated on both those sides, but differently) and the "new count" (messages that are translated on this side but not on the other side). We'll still have to give these better names etc. At least I hope that the new calculations make this wart a bit more explicit.
Most of the branch is tests for the desired behaviour. Statistics were already tested, but those tests were written under the assumptions of the old model. At the very least there's one extra dimension to test for, and I wanted to be able to formulate exactly what behaviour I want from the new code, from scratch, yet not lose sight of what might change in old-model code that's been minimally adapted to the new model.
Our data model can get quite complex, although overall the Recife change is probably a simplification. Among the dimensions that tests (and production code) need to deal with are:
* POTMsgSets can be obsolete in a POTemplate (i.e. TranslationTemplateItem.sequence = 0).
* There's POTMsgSets that have a plural form vs. single-form ones.
* TranslationMessages can be shared, or diverged to your POTemplate, or diverged to another one.
* The two "translation sides": Ubuntu vs. upstream.
* TranslationMessages can be current on the side you're looking at, or suggestions that haven't been reviewed, or suggestions that have been reviewed.
* TranslationMessages can be shared and current on the other side, or current but diverged, or not current at all.
* TranslationMessages can be blank (so present but untranslated nonetheless), incomplete (not implementing all plural forms required by the language), or complete (fully translated).
In this branch I'm not dealing with incomplete ones yet. There's a TODO to that effect.
To test,
{{{
./bin/test -vvc lp.translations.tests.test_pofile -t Statistics
}}}
No lint,
Jeroen
--
https://code.launchpad.net/~jtv/launchpad/recife-statistics/+merge/42080
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~jtv/launchpad/recife-statistics into lp:launchpad/db-devel.
=== modified file 'BRANCH.TODO'
--- BRANCH.TODO 2010-11-04 09:32:28 +0000
+++ BRANCH.TODO 2010-11-29 08:20:32 +0000
@@ -2,3 +2,9 @@
# landing. There is a test to ensure it is empty in trunk. If there is
# stuff still here when you are ready to land, the items should probably
# be converted to bugs so they can be scheduled.
+Clean out old statistics calculation methods.
+Clean out old tests.
+Test that all counts ignore diverged messages for other templates (this side).
+Test that all counts ignore diverged messages for other templates (other side).
+Test statistics for treatment of incomplete messages.
+Fix up code to treat incomplete messages as untranslated.
=== modified file 'lib/lp/translations/interfaces/pofile.py'
--- lib/lp/translations/interfaces/pofile.py 2010-11-21 20:46:19 +0000
+++ lib/lp/translations/interfaces/pofile.py 2010-11-29 08:20:32 +0000
@@ -386,11 +386,14 @@
if the end of the table has been reached.
"""
- def getPOFilesWithTranslationCredits():
- """Get all POFiles with potential translation credits messages.
+ def getPOFilesWithTranslationCredits(untranslated=False):
+ """Get POFiles with potential translation credits messages.
Returns a ResultSet of (POFile, POTMsgSet) tuples, ordered by
POFile.id.
+
+ :param untranslated: Look only for `POFile`s with a credits
+ message that is not translated.
"""
def getPOFilesTouchedSince(date):
=== modified file 'lib/lp/translations/model/pofile.py'
--- lib/lp/translations/model/pofile.py 2010-11-26 05:10:27 +0000
+++ lib/lp/translations/model/pofile.py 2010-11-29 08:20:32 +0000
@@ -785,7 +785,8 @@
'(POTMsgSet.msgid_plural IS NULL OR (%s))' % plurals_query)
return query
- def updateStatistics(self):
+ def old_updateStatistics(self):
+ raise AssertionError("This is dead code.")
"""See `IPOFile`."""
# make sure all the data is in the db
flush_database_updates()
@@ -841,13 +842,132 @@
self.unreviewed_count = unreviewed
return self.getStatistics()
+ def _countTranslations(self):
+ """Count `currentcount`, `updatescount`, and `rosettacount`."""
+ side_traits = getUtility(ITranslationSideTraitsSet).getForTemplate(
+ self.potemplate)
+ coalesce_other_msgstrs = "COALESCE(%s)" % ", ".join([
+ "Other.msgstr%d" % form
+ for form in xrange(TranslationConstants.MAX_PLURAL_FORMS)])
+ params = {
+ 'potemplate': quote(self.potemplate),
+ 'language': quote(self.language),
+ 'flag': side_traits.flag_name,
+ 'other_flag': side_traits.other_side_traits.flag_name,
+ 'coalesce_other_msgstrs': coalesce_other_msgstrs,
+ }
+ query = """
+ SELECT other_msgstrs, same_on_both_sides, count(*)
+ FROM (
+ SELECT
+ DISTINCT ON (TTI.potmsgset)
+ %(coalesce_other_msgstrs)s AS other_msgstrs,
+ (Other.id = Current.id) AS same_on_both_sides
+ FROM TranslationTemplateItem AS TTI
+ JOIN TranslationMessage AS Current ON
+ Current.potmsgset = TTI.potmsgset AND
+ Current.language = %(language)s AND
+ COALESCE(Current.potemplate, %(potemplate)s) =
+ %(potemplate)s AND
+ Current.%(flag)s IS TRUE
+ LEFT OUTER JOIN TranslationMessage AS Other ON
+ Other.potmsgset = TTI.potmsgset AND
+ Other.language = %(language)s AND
+ Other.%(other_flag)s IS TRUE AND
+ Other.potemplate IS NULL AND
+ Other.potemplate IS NULL
+ WHERE
+ TTI.potemplate = %(potemplate)s AND
+ TTI.sequence > 0
+ ORDER BY
+ TTI.potmsgset,
+ Current.potemplate NULLS LAST
+ ) AS translated_messages
+ GROUP BY other_msgstrs, same_on_both_sides
+ """ % params
+
+ this_side_only = 0
+ translated_differently = 0
+ translated_same = 0
+ for row in IStore(self).execute(query):
+ (other_msgstrs, same_on_both_sides, count) = row
+ if other_msgstrs is None:
+ this_side_only += count
+ elif same_on_both_sides:
+ translated_same += count
+ else:
+ translated_differently += count
+
+ return (
+ translated_same,
+ translated_differently,
+ translated_differently + this_side_only,
+ )
+
+ def _countNewSuggestions(self):
+ """Count messages with new suggestions."""
+ flag_name = getUtility(ITranslationSideTraitsSet).getForTemplate(
+ self.potemplate).flag_name
+ suggestion_nonempty = "COALESCE(%s) IS NOT NULL" % ', '.join([
+ 'Suggestion.msgstr%d' % form
+ for form in xrange(TranslationConstants.MAX_PLURAL_FORMS)])
+ params = {
+ 'language': quote(self.language),
+ 'potemplate': quote(self.potemplate),
+ 'flag': flag_name,
+ 'suggestion_nonempty': suggestion_nonempty,
+ }
+ query = """
+ SELECT count(*)
+ FROM (
+ SELECT DISTINCT ON (TTI.potmsgset) *
+ FROM TranslationTemplateItem TTI
+ LEFT OUTER JOIN TranslationMessage AS Current ON
+ Current.potmsgset = TTI.potmsgset AND
+ Current.language = %(language)s AND
+ COALESCE(Current.potemplate, %(potemplate)s) =
+ %(potemplate)s AND
+ Current.%(flag)s IS TRUE
+ WHERE
+ TTI.potemplate = %(potemplate)s AND
+ TTI.sequence <> 0 AND
+ EXISTS (
+ SELECT *
+ FROM TranslationMessage Suggestion
+ WHERE
+ Suggestion.potmsgset = TTI.potmsgset AND
+ Suggestion.language = %(language)s AND
+ Suggestion.%(flag)s IS FALSE AND
+ %(suggestion_nonempty)s AND
+ Suggestion.date_created > COALESCE(
+ Current.date_reviewed,
+ Current.date_created,
+ TIMESTAMP 'epoch')
+ )
+ ORDER BY TTI.potmsgset, Current.potemplate NULLS LAST
+ ) AS messages_with_suggestions
+ """ % params
+ for count, in IStore(self).execute(query):
+ return count
+
+ def updateStatistics(self):
+ """See `IPOFile`."""
+ if self.potemplate.messageCount() == 0:
+ self.potemplate.updateMessageCount()
+
+ (
+ self.currentcount,
+ self.updatescount,
+ self.rosettacount,
+ ) = self._countTranslations()
+ self.unreviewed_count = self._countNewSuggestions()
+ return self.getStatistics()
+
def updateHeader(self, new_header):
"""See `IPOFile`."""
if new_header is None:
return
- # XXX sabdfl 2005-05-27 should we also differentiate between
- # washeaderfuzzy and isheaderfuzzy?
self.topcomment = new_header.comment
self.header = new_header.getRawContent()
self.fuzzyheader = new_header.is_fuzzy
@@ -1424,7 +1544,8 @@
TranslationTemplateItem.potemplateID == POFile.potemplateID,
POTMsgSet.id == TranslationTemplateItem.potmsgsetID,
POTMsgSet.msgid_singular == POMsgID.id,
- POMsgID.msgid.is_in(POTMsgSet.credits_message_ids)]
+ POMsgID.msgid.is_in(POTMsgSet.credits_message_ids),
+ ]
if untranslated:
message_select = Select(
True,
=== modified file 'lib/lp/translations/model/potemplate.py'
--- lib/lp/translations/model/potemplate.py 2010-11-21 20:46:19 +0000
+++ lib/lp/translations/model/potemplate.py 2010-11-29 08:20:32 +0000
@@ -533,6 +533,10 @@
"""See `IRosettaStats`."""
return self.messagecount
+ def updateMessageCount(self):
+ """Update `self.messagecount`."""
+ self.messagecount = self.getPOTMsgSetsCount()
+
def currentCount(self, language=None):
"""See `IRosettaStats`."""
if language is None:
@@ -944,7 +948,7 @@
flush_database_updates()
# Update cached number of msgsets.
- self.messagecount = self.getPOTMsgSetsCount()
+ self.updateMessageCount()
# The upload affects the statistics for all translations of this
# template. Recalculate those as well. This takes time and
=== modified file 'lib/lp/translations/tests/test_pofile.py'
--- lib/lp/translations/tests/test_pofile.py 2010-11-26 05:10:27 +0000
+++ lib/lp/translations/tests/test_pofile.py 2010-11-29 08:20:32 +0000
@@ -20,13 +20,12 @@
from canonical.database.constants import UTC_NOW
from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities
from canonical.launchpad.webapp.publisher import canonical_url
-from canonical.testing.layers import (
- LaunchpadZopelessLayer,
- ZopelessDatabaseLayer,
- )
+from canonical.testing.layers import ZopelessDatabaseLayer
from lp.app.enums import ServiceUsage
from lp.testing import TestCaseWithFactory
+from lp.testing.fakemethod import FakeMethod
from lp.translations.interfaces.pofile import IPOFileSet
+from lp.translations.interfaces.side import ITranslationSideTraitsSet
from lp.translations.interfaces.translatablemessage import (
ITranslatableMessage,
)
@@ -814,6 +813,16 @@
self.devel_pofile.getPOTMsgSetChangedInUbuntu())
self.assertEquals(found_translations, [self.potmsgset])
+ def test_messageCount(self):
+ # POFile.messageCount just forwards to POTmeplate.messageCount.
+ pofile = removeSecurityProxy(self.factory.makePOFile())
+ pofile.potemplate.messageCount = FakeMethod(result=99)
+ self.assertEqual(99, pofile.messageCount())
+
+ def test_initial_statistics_consistency(self):
+ # A `POFile` starts out with consistent statistics.
+ self.assertTrue(self.factory.makePOFile().testStatistics())
+
def test_updateStatistics(self):
# Test that updating statistics keeps working.
@@ -837,10 +846,11 @@
# Third POTMsgSet is translated, and with a suggestion.
potmsgset = self.factory.makePOTMsgSet(self.devel_potemplate)
potmsgset.setSequence(self.devel_potemplate, 3)
- self.factory.makeTranslationMessage(
+ update_date = datetime.now(pytz.UTC) - timedelta(1)
+ self.factory.makeCurrentTranslationMessage(
pofile=self.devel_pofile, potmsgset=potmsgset,
- translations=[u"Translation"],
- date_updated=datetime.now(pytz.UTC)-timedelta(1))
+ translations=[u"Translation"], date_created=update_date,
+ date_reviewed=update_date)
self.factory.makeSuggestion(
pofile=self.devel_pofile, potmsgset=potmsgset,
translations=[u"Another suggestion"])
@@ -848,26 +858,26 @@
# Fourth POTMsgSet is translated in import.
potmsgset = self.factory.makePOTMsgSet(self.devel_potemplate)
potmsgset.setSequence(self.devel_potemplate, 4)
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.devel_pofile, potmsgset=potmsgset,
- translations=[u"Imported translation"], is_current_upstream=True)
+ translations=[u"Imported translation"], current_other=True)
# Fifth POTMsgSet is translated in import, but changed in Ubuntu.
potmsgset = self.factory.makePOTMsgSet(self.devel_potemplate)
potmsgset.setSequence(self.devel_potemplate, 5)
- self.factory.makeTranslationMessage(
- pofile=self.devel_pofile, potmsgset=potmsgset,
- translations=[u"Imported translation"], is_current_upstream=True)
- translation = self.factory.makeTranslationMessage(
- pofile=self.devel_pofile, potmsgset=potmsgset,
- translations=[u"LP translation"], is_current_upstream=False)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=self.devel_pofile, potmsgset=potmsgset,
+ translations=[u"Imported translation"], current_other=True)
+ translation = self.factory.makeCurrentTranslationMessage(
+ pofile=self.devel_pofile, potmsgset=potmsgset,
+ translations=[u"LP translation"], current_other=False)
# Sixth POTMsgSet is translated in LP only.
potmsgset = self.factory.makePOTMsgSet(self.devel_potemplate)
potmsgset.setSequence(self.devel_potemplate, 6)
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.devel_pofile, potmsgset=potmsgset,
- translations=[u"New translation"], is_current_upstream=False)
+ translations=[u"New translation"], current_other=False)
removeSecurityProxy(self.devel_potemplate).messagecount = (
self.devel_potemplate.getPOTMsgSetsCount())
@@ -1111,14 +1121,14 @@
imported_credits_text = u"Imported Contributor <name@xxxxxxxxxxx>"
# Import a translation credits message to 'translator-credits'.
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.credits_potmsgset,
translations=[imported_credits_text],
- is_current_upstream=True)
+ current_other=True)
# `person` updates the translation using Launchpad.
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translator=person)
@@ -1131,11 +1141,11 @@
credits_text)
# Now, re-import this generated message.
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.credits_potmsgset,
translations=[credits_text],
- is_current_upstream=True)
+ current_other=True)
credits_text = self.pofile.prepareTranslationCredits(
self.credits_potmsgset)
@@ -1405,7 +1415,7 @@
class TestPOFileSet(TestCaseWithFactory):
"""Test PO file set methods."""
- layer = LaunchpadZopelessLayer
+ layer = ZopelessDatabaseLayer
def setUp(self):
# Create a POFileSet to work with.
@@ -1643,47 +1653,30 @@
self.pofileset.getPOFilesWithTranslationCredits()))
def test_getPOFilesWithTranslationCredits_untranslated(self):
- # We need absolute DB access to be able to remove a translation
- # message.
- LaunchpadZopelessLayer.switchDbUser('postgres')
+ # With "untranslated=True," getPOFilesWithTranslationCredits
+ # looks for POFiles whose translation credits messages are
+ # untranslated.
- # Initially, we only get data from the sampledata, all of which
- # are untranslated.
- sampledata_pofiles = list(
+ # The sample data may contain some matching POFiles, but we'll
+ # ignore those.
+ initial_matches = set(
self.pofileset.getPOFilesWithTranslationCredits(
untranslated=True))
- total = len(sampledata_pofiles)
- self.assertEquals(3, total)
- # All POFiles with translation credits messages are
- # returned along with relevant POTMsgSets.
- potemplate1 = self.factory.makePOTemplate()
+ potemplate = self.factory.makePOTemplate()
credits_potmsgset = self.factory.makePOTMsgSet(
- potemplate1, singular=u'translator-credits', sequence=1)
-
- sr_pofile = self.factory.makePOFile('sr', potemplate=potemplate1)
- pofiles_with_credits = (
- self.pofileset.getPOFilesWithTranslationCredits(
- untranslated=True))
- self.assertNotIn((sr_pofile, credits_potmsgset),
- list(pofiles_with_credits))
- self.assertEquals(
- total,
- pofiles_with_credits.count())
-
- # Removing a translation for this message, removes it
- # from a result set when untranslated=True is passed in.
- message = credits_potmsgset.getSharedTranslationMessage(
- sr_pofile.language)
- message.destroySelf()
- pofiles_with_credits = (
- self.pofileset.getPOFilesWithTranslationCredits(
- untranslated=True))
- self.assertIn((sr_pofile, credits_potmsgset),
- list(pofiles_with_credits))
- self.assertEquals(
- total + 1,
- pofiles_with_credits.count())
+ potemplate, singular=u'translator-credits', sequence=1)
+ pofile = self.factory.makePOFile(potemplate=potemplate)
+
+ credits_translation = credits_potmsgset.getCurrentTranslation(
+ potemplate, pofile.language, potemplate.translation_side)
+ credits_translation.is_current_ubuntu = False
+ credits_translation.is_current_upstream = False
+
+ self.assertEqual(
+ initial_matches.union([(pofile, credits_potmsgset)]),
+ set(self.pofileset.getPOFilesWithTranslationCredits(
+ untranslated=True)))
def test_getPOFilesByPathAndOrigin_path_mismatch(self):
# getPOFilesByPathAndOrigin matches on POFile path.
@@ -1808,11 +1801,11 @@
self.assertEquals(self.pofile.currentCount(), 0)
# Adding an imported translation increases currentCount().
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translations=["Imported current"],
- is_current_upstream=True)
+ current_other=True)
self.pofile.updateStatistics()
self.assertEquals(self.pofile.currentCount(), 1)
@@ -1834,7 +1827,7 @@
# Adding a current translation for an untranslated
# message increases the count of new translations in LP.
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translations=["Current"])
@@ -1845,16 +1838,16 @@
# If we get an 'imported' translation for what
# we already have as 'new', it's not considered 'new'
# anymore since it has been synced.
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translations=["Current"])
- # Reimport it but with is_current_upstream=True.
- self.factory.makeTranslationMessage(
+ # Reimport it but with is_current_ubuntu=True.
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translations=["Current"],
- is_current_upstream=True)
+ current_other=True)
self.pofile.updateStatistics()
self.assertEquals(self.pofile.newCount(), 0)
@@ -1863,12 +1856,12 @@
# If we change an 'imported' translation through
# Launchpad, it's still not considered 'new',
# but an 'update' instead.
- self.factory.makeTranslationMessage(
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translations=["Imported"],
- is_current_upstream=True)
- self.factory.makeTranslationMessage(
+ current_other=True)
+ self.factory.makeCurrentTranslationMessage(
pofile=self.pofile,
potmsgset=self.potmsgset,
translations=["Changed"])
@@ -2412,3 +2405,401 @@
self.getTranslationPillar().translationgroup = group
self.assertFalse(self.pofile.canEditTranslations(group.owner))
self.assertFalse(self.pofile.canAddSuggestions(group.owner))
+
+
+class StatisticsTestScenario:
+ """Test case mixin: `POFile` statistics."""
+ layer = ZopelessDatabaseLayer
+
+ def makePOFile(self):
+ """Create a `POFile` to run statistics tests against."""
+ raise NotImplementedError("makePOFile")
+
+ def _getSideTraits(self, potemplate):
+ """Return `TranslationSideTraits` for `potemplate`."""
+ return getUtility(ITranslationSideTraitsSet).getForTemplate(
+ potemplate)
+
+ def _makeOtherSideTranslation(self, pofile, potmsgset=None,
+ translations=None):
+ """Create a current `TranslationMessage` for the other side."""
+ message = self.factory.makeSuggestion(
+ pofile=pofile, potmsgset=potmsgset, translations=translations)
+ traits = self._getSideTraits(pofile.potemplate)
+ traits.other_side_traits.setFlag(message, True)
+ return message
+
+ def test_statistics_are_initialized_correctly(self):
+ # When a POFile is created, its statistics are initialized as if
+ # they had been freshly updated.
+ pofile = self.makePOFile()
+ stats = pofile.getStatistics()
+ pofile.updateStatistics()
+ self.assertEqual(stats, pofile.getStatistics())
+
+ def test_translatedCount_initial(self):
+ pofile = self.makePOFile()
+ self.assertEqual(0, pofile.translatedCount())
+
+ def test_translatedCount_potmsgset_initial(self):
+ pofile = self.makePOFile()
+ self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.translatedCount())
+
+ def test_translatedCount(self):
+ # Making a translation message current increases the POFile's
+ # translatedCount.
+ pofile = self.makePOFile()
+ suggestion = self.factory.makeCurrentTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.translatedCount())
+
+ def test_translatedCount_ignores_obsolete(self):
+ # Translations of obsolete POTMsgSets do not count as
+ # translated.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=0)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.translatedCount())
+
+ def test_translatedCount_other_side(self):
+ # Translations on the other side do not count as translated.
+ pofile = self.makePOFile()
+ self._makeOtherSideTranslation(pofile)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.translatedCount())
+
+ def test_translatedCount_diverged(self):
+ # Diverged translations are also counted.
+ pofile = self.makePOFile()
+ diverged = self.factory.makeDivergedTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.translatedCount())
+
+ def test_translatedCount_ignores_masked_shared_translations(self):
+ # A shared current translation that is masked by a diverged one
+ # is not counted.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ self.factory.makeDivergedTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.translatedCount())
+
+ def test_untranslatedCount_potmsgset_initial(self):
+ pofile = self.makePOFile()
+ self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.untranslatedCount())
+
+ def test_untranslatedCount_initial(self):
+ self.assertEqual(0, self.makePOFile().untranslatedCount())
+
+ def test_untranslatedCount(self):
+ # Translating a message removes it from the untranslatedCount.
+ pofile = self.makePOFile()
+ self.factory.makeCurrentTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.untranslatedCount())
+
+ def test_untranslatedCount_ignores_obsolete(self):
+ # Translations of obsolete POTMsgSets do not count as
+ # untranslated.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=0)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.untranslatedCount())
+
+ def test_untranslatedCount_other_side(self):
+ # Messages that are translated on the other side can still be in
+ # the untranslatedCount.
+ pofile = self.makePOFile()
+ self._makeOtherSideTranslation(pofile)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.untranslatedCount())
+
+ def test_untranslatedCount_diverged(self):
+ # Diverged translations are also counted.
+ pofile = self.makePOFile()
+ diverged = self.factory.makeDivergedTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.untranslatedCount())
+
+ def test_untranslatedCount_ignores_masked_shared_translations(self):
+ # A shared current translation that is masked by a diverged one
+ # is only subtracted from the untranslatedCount once.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ self.factory.makeDivergedTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.untranslatedCount())
+
+ def test_currentCount_initial(self):
+ self.assertEqual(0, self.makePOFile().currentCount())
+
+ def test_currentCount_potmsgset_initial(self):
+ pofile = self.makePOFile()
+ self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.currentCount())
+
+ def test_currentCount(self):
+ # A translation that is shared between Ubuntu and upstream is
+ # counted in currentCount.
+ pofile = self.makePOFile()
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, current_other=True)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.currentCount())
+
+ def test_currentCount_ignores_obsolete(self):
+ # The currentCount does not include obsolete messages.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=0)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset, current_other=True)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.currentCount())
+
+ def test_currentCount_ignores_onesided_translation(self):
+ # A translation that is only current on one side is not included
+ # in currentCount.
+ pofile = self.makePOFile()
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, current_other=False)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.currentCount())
+
+ def test_currentCount_different(self):
+ # A message that is translated differently in Ubuntu than
+ # upstream is not included in currentCount.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self._makeOtherSideTranslation(pofile, potmsgset=potmsgset)
+ this_translation = self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.currentCount())
+
+ def test_currentCount_diverged(self):
+ # Diverging from a translation that's shared between Ubuntu and
+ # upstream decrements the currentCount.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset, current_other=True)
+ self.factory.makeDivergedTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.currentCount())
+
+ def test_rosettaCount_initial(self):
+ self.assertEqual(0, self.makePOFile().rosettaCount())
+
+ def test_rosettaCount_potmsgset_initial(self):
+ pofile = self.makePOFile()
+ self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.rosettaCount())
+
+ def test_rosettaCount(self):
+ # rosettaCount counts messages that are translated on this side
+ # but not the other side.
+ pofile = self.makePOFile()
+ self.factory.makeCurrentTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.rosettaCount())
+
+ def test_rosettaCount_ignores_obsolete(self):
+ # The rosettaCount ignores obsolete messages.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=0)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.rosettaCount())
+
+ def test_rosettaCount_diverged(self):
+ # Diverged messages are also counted towards the rosettaCount.
+ pofile = self.makePOFile()
+ self.factory.makeDivergedTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.rosettaCount())
+
+ def test_rosettaCount_ignores_shared_messages(self):
+ # Messages that are shared with the other side are not part of
+ # the rosettaCount.
+ pofile = self.makePOFile()
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, current_other=True)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.rosettaCount())
+
+ def test_rosettaCount_ignores_messages_translated_on_other_side(self):
+ # Messages that are translated on the other side but not on this
+ # one do not count towards the rosettaCount.
+ pofile = self.makePOFile()
+ self._makeOtherSideTranslation(pofile)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.rosettaCount())
+
+ def test_rosettaCount_includes_different_translations(self):
+ # The rosettaCount does include messages that are translated
+ # differently on the two sides.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self._makeOtherSideTranslation(pofile, potmsgset=potmsgset)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.rosettaCount())
+
+ def test_updatesCount_initial(self):
+ self.assertEqual(0, self.makePOFile().updatesCount())
+
+ def test_updatesCount_potmsgset_initial(self):
+ pofile = self.makePOFile()
+ self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.updatesCount())
+
+ def test_updatesCount(self):
+ # The updatesCount counts messages that are translated on the
+ # other side, but differently.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self._makeOtherSideTranslation(pofile, potmsgset=potmsgset)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.updatesCount())
+
+ def test_updatesCount_ignores_obsolete(self):
+ # The updatesCount ignores obsolete messages.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=0)
+ self._makeOtherSideTranslation(pofile, potmsgset=potmsgset)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.updatesCount())
+
+ def test_updatesCount_diverged(self):
+ # Diverged messages can be part of the updatesCount.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset, current_other=True)
+ self.factory.makeDivergedTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.updatesCount())
+
+ def test_updatesCount_diverged_ignores_untranslated_other(self):
+ # Diverged messages are not part of the updatesCount if there is
+ # no translation on the other side; they fall under rosettaCount.
+ pofile = self.makePOFile()
+ self.factory.makeDivergedTranslationMessage(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.updatesCount())
+
+ def test_unreviewedCount_initial(self):
+ self.assertEqual(0, self.makePOFile().unreviewedCount())
+
+ def test_unreviewedCount_potmsgset_initial(self):
+ pofile = self.makePOFile()
+ self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.unreviewedCount())
+
+ def test_unreviewedCount(self):
+ # A completely untranslated message with a suggestion counts as
+ # unreviewed.
+ pofile = self.makePOFile()
+ self.factory.makeSuggestion(pofile=pofile)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.unreviewedCount())
+
+ def test_unreviewedCount_ignores_obsolete(self):
+ # The unreviewedCount ignores obsolete messages.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=0)
+ self.factory.makeSuggestion(pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.unreviewedCount())
+
+ def test_unreviewedCount_counts_msgids_not_suggestions(self):
+ # The unreviewedCount counts messages with unreviewed
+ # suggestions, not the suggestions themselves.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ self.factory.makeSuggestion(pofile=pofile, potmsgset=potmsgset)
+ self.factory.makeSuggestion(pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.unreviewedCount())
+
+ def test_unreviewedCount_ignores_reviewed_suggestions(self):
+ # In order to affect the unreviewedCount, a suggestion has to be
+ # newer than the review date on the current translation.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ suggestion = self.factory.makeSuggestion(
+ pofile=pofile, potmsgset=potmsgset)
+ translation = self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(0, pofile.unreviewedCount())
+
+ def test_unreviewedCount_includes_new_suggestions(self):
+ # Suggestions that are newer than the review date om the current
+ # translation are included in the unreviewedCount.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ translation = self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ translation.date_reviewed -= timedelta(1)
+ suggestion = self.factory.makeSuggestion(
+ pofile=pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.unreviewedCount())
+
+ def test_unreviewedCount_includes_other_side_translation(self):
+ # A translation on the other side that's newer than the review
+ # date on the current translation on this side also counts as an
+ # unreviewed suggestion on this side.
+ pofile = self.makePOFile()
+ potmsgset = self.factory.makePOTMsgSet(pofile.potemplate, sequence=1)
+ this_translation = self.factory.makeCurrentTranslationMessage(
+ pofile=pofile, potmsgset=potmsgset)
+ this_translation.date_reviewed -= timedelta(1)
+ other_translation = self._makeOtherSideTranslation(
+ pofile, potmsgset=potmsgset)
+ pofile.updateStatistics()
+ self.assertEqual(1, pofile.unreviewedCount())
+
+
+class TestUpstreamStatistics(StatisticsTestScenario, TestCaseWithFactory):
+ """Test statistics on upstream `POFile`s."""
+
+ def makePOFile(self):
+ return self.factory.makePOFile()
+
+
+class TestUbuntuStatistics(StatisticsTestScenario, TestCaseWithFactory):
+ """Test statistics on Ubuntu `POFile`s."""
+
+ def makePOFile(self):
+ package = self.factory.makeSourcePackage()
+ return self.factory.makePOFile(
+ potemplate=self.factory.makePOTemplate(
+ distroseries=package.distroseries,
+ sourcepackagename=package.sourcepackagename))
