launchpad-reviewers team mailing list archive
-
launchpad-reviewers team
-
Mailing list archive
-
Message #02071
[Merge] lp:~jtv/launchpad/recife-statistics-aftershock into lp:launchpad/db-devel
Jeroen T. Vermeulen has proposed merging lp:~jtv/launchpad/recife-statistics-aftershock into lp:launchpad/db-devel with lp:~jtv/launchpad/recife-statistics as a prerequisite.
Requested reviews:
Launchpad code reviewers (launchpad-reviewers): code
Follow-up to the branch that re-does translation statistics for the new "Recife" data model.
That branch broke some tests that relied on either a mix of the old and the new model, or on sample data. The latter are particularly brittle, and I weakened those tests a bit to avoid that. The POFile views in particular are either covered by more modern unit tests (some of which I also had to fix up) or slated to be replaced by new views.
Some of the doctests (getPOTMsgSetUntranslated etc.) are documentation to complement unit tests. In practice these tests are generally useless, yet hard to maintain because of all the state that is carried along throughout a doctest. A lot of the diff is removal of such tests, and compensating for the effects that this has on subsequent tests that should be related.
We'll Q/A this with the rest of the Recife feature branch. I cleaned up lint where I could.
Jeroen
--
https://code.launchpad.net/~jtv/launchpad/recife-statistics-aftershock/+merge/42274
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~jtv/launchpad/recife-statistics-aftershock into lp:launchpad/db-devel.
=== modified file 'lib/lp/translations/browser/tests/pofile-views.txt'
--- lib/lp/translations/browser/tests/pofile-views.txt 2010-11-16 13:10:43 +0000
+++ lib/lp/translations/browser/tests/pofile-views.txt 2010-11-30 16:59:29 +0000
@@ -169,16 +169,11 @@
But first, let's see current values.
- >>> pofile_es.updateStatistics()
- (6, 1, 1, 1)
- >>> pofile_es.currentCount()
- 6
+ >>> stats = pofile_es.updateStatistics()
>>> pofile_es.updatesCount()
- 1
+ 0
>>> pofile_es.rosettaCount()
- 1
- >>> pofile_es.unreviewedCount()
- 1
+ 7
Now we do a submission with new translations:
@@ -208,8 +203,11 @@
And check again.
- >>> pofile_es.updateStatistics()
- (5, 2, 3, 1)
+ >>> stats = pofile_es.updateStatistics()
+ >>> pofile_es.updatesCount()
+ 1
+ >>> pofile_es.rosettaCount()
+ 8
The messages displayed on the +translate page are always in ascending order of
their POTMsgSets' sequence numbers.
=== modified file 'lib/lp/translations/browser/tests/test_pofile_view.py'
--- lib/lp/translations/browser/tests/test_pofile_view.py 2010-10-18 16:36:46 +0000
+++ lib/lp/translations/browser/tests/test_pofile_view.py 2010-11-30 16:59:29 +0000
@@ -47,24 +47,29 @@
# A translated message.
self.translated = self.factory.makePOTMsgSet(
self.potemplate, sequence=2)
- self.factory.makeTranslationMessage(self.pofile, self.translated)
+ self.factory.makeCurrentTranslationMessage(
+ self.pofile, self.translated)
# A translated message with a new suggestion.
self.new_suggestion = self.factory.makePOTMsgSet(
self.potemplate, sequence=3)
- self.factory.makeTranslationMessage(
- self.pofile, self.new_suggestion,
- date_updated=self.now())
+ now = self.now()
+ self.factory.makeCurrentTranslationMessage(
+ pofile=self.pofile, potmsgset=self.new_suggestion,
+ date_created=now, date_reviewed=now)
self.factory.makeSuggestion(
- self.pofile, self.new_suggestion, date_created=self.now())
+ pofile=self.pofile, potmsgset=self.new_suggestion,
+ date_created=self.now())
# An upstream that was changed in Ubuntu.
- self.changed = self.factory.makePOTMsgSet(
- self.potemplate, sequence=4)
- self.factory.makeTranslationMessage(
- self.pofile, self.changed, is_current_upstream=True,
- date_updated=self.now())
- self.factory.makeTranslationMessage(
- self.pofile, self.changed,
- date_updated=self.now())
+ self.changed = self.factory.makePOTMsgSet(self.potemplate, sequence=4)
+ now = self.now()
+ ubuntu_translation = self.factory.makeSuggestion(
+ pofile=self.pofile, potmsgset=self.changed,
+ date_created=self.now())
+ ubuntu_translation.is_current_ubuntu = True
+ now = self.now()
+ self.factory.makeCurrentTranslationMessage(
+ pofile=self.pofile, potmsgset=self.changed, date_created=now,
+ date_reviewed=now)
# Update statistics so that shown_count returns correct values.
self.pofile.updateStatistics()
@@ -196,8 +201,7 @@
def _useNonnewTranslator(self):
"""Create a user who's done translations, and log in as that user."""
user = self._makeLoggedInUser()
- self.factory.makeSharedTranslationMessage(
- translator=user, suggestion=True)
+ self.factory.makeSuggestion(translator=user)
return user
def _makeView(self, pofile=None, request=None):
@@ -431,4 +435,3 @@
DocumentationScenarioMixin):
layer = ZopelessDatabaseLayer
view_class = POFileTranslateView
-
=== modified file 'lib/lp/translations/doc/pofile-verify-stats.txt'
--- lib/lp/translations/doc/pofile-verify-stats.txt 2010-05-07 12:29:23 +0000
+++ lib/lp/translations/doc/pofile-verify-stats.txt 2010-11-30 16:59:29 +0000
@@ -20,13 +20,16 @@
it directly.
>>> import transaction
+ >>> from lp.services.mail.stub import test_emails
>>> from lp.translations.scripts.verify_pofile_stats import (
... VerifyPOFileStatsProcess)
>>> from canonical.launchpad.scripts.logger import FakeLogger
>>> logger = FakeLogger()
>>> VerifyPOFileStatsProcess(transaction, logger).run()
INFO Starting verification of POFile stats at id 0
+ ...
INFO Done.
+ >>> old_email = test_emails.pop()
All POFile statistics in our database are now correct, and we can test in
more detail.
@@ -92,7 +95,6 @@
The Translations administrators also receive an email about the error.
- >>> from lp.services.mail.stub import test_emails
>>> from_addr, to_addrs, body = test_emails.pop()
>>> len(test_emails)
0
=== modified file 'lib/lp/translations/doc/pofile.txt'
--- lib/lp/translations/doc/pofile.txt 2010-11-18 18:19:41 +0000
+++ lib/lp/translations/doc/pofile.txt 2010-11-30 16:59:29 +0000
@@ -261,7 +261,8 @@
>>> new_header_string = '''Project-Id-Version: es
... POT-Creation-Date: 2004-08-18 11:10+0200
... PO-Revision-Date: 2005-08-18 13:22+0000
- ... Last-Translator: Carlos Perell\xc3\xb3 Mar\xc3\xadn <carlos@xxxxxxxxxxxxx>
+ ... Last-Translator: Carlos Perell\xc3\xb3 Mar\xc3\xadn
+ ... <carlos@xxxxxxxxxxxxx>
... Language-Team: Spanish <traductores@xxxxxxxxxxxx>
... MIME-Version: 1.0
... Content-Type: text/plain; charset=UTF-8
@@ -452,7 +453,9 @@
... 'prefs.js' in stream_list[i]):
... break
>>> stream_list[i]
- 'msgstr "\xc0\xdf\xc4\xea\xa4\xce\xa5\xab\xa5\xb9\xa5\xbf\xa5\xde\xa5\xa4\xa5\xba\xa4\xcb /etc/mozilla/prefs.js \xa4\xac\xcd\xf8\xcd\xd1\xa4\xc7\xa4\xad\xa4\xde\xa4\xb9\xa1\xa3"'
+ 'msgstr "\xc0\xdf\xc4\xea\xa4\xce\xa5\xab...\xa5\xba\xa4\xcb
+ /etc/mozilla/prefs.js
+ \xa4\xac\xcd\xf8\xcd\xd1\xa4\xc7\xa4\xad\xa4\xde\xa4\xb9\xa1\xa3"'
Now, let's force the UTF-8 encoding.
@@ -473,7 +476,9 @@
... 'prefs.js' in stream_list[i]):
... break
>>> stream_list[i]
- 'msgstr "\xe8\xa8\xad\xe5\xae\x9a\xe3\x81\xae\xe3\x82\xab\xe3\x82\xb9\xe3\x82\xbf\xe3\x83\x9e\xe3\x82\xa4\xe3\x82\xba\xe3\x81\xab /etc/mozilla/prefs.js \xe3\x81\x8c\xe5\x88\xa9\xe7\x94\xa8\xe3\x81\xa7\xe3\x81\x8d\xe3\x81\xbe\xe3\x81\x99\xe3\x80\x82"'
+ 'msgstr "\xe8\xa8\xad\xe5\xae\x9a\xe3\x81...\xba\xe3\x81\xab
+ /etc/mozilla/prefs.js
+ \xe3\x81\x8c\xe5\x88\xa9\xe7\x94\xa8\xe3...\x99\xe3\x80\x82"'
There are some situations when a msgid_plural changes, while the msgid
singular remains unchanged.
@@ -522,22 +527,6 @@
...
-getPOTMsgSetUntranslated
-------------------------
-
-The POTMsgSet that are untranslated are the ones that are not complete.
-
- >>> untranslated_potmsgset = pofile_es.getPOTMsgSetUntranslated()
- >>> untranslated_potmsgset.count()
- 15
-
- >>> for potmsgset in untranslated_potmsgset:
- ... pomsgset = potmsgset.getCurrentTranslationMessage(
- ... pofile_es.potemplate, pofile_es.language)
- ... assert (pomsgset is None or
- ... (pomsgset.iscomplete == False))
-
-
createMessageSetFromText
------------------------
@@ -590,271 +579,6 @@
[None, None, None]
-getPOTMsgSetChangedInUbuntu
----------------------------
-
-This method returns only those POTMsgSet's which contain translation
-submissions through Launchpad for Ubuntu which change existing upstream
-translations.
-
-Lets get Evolution Spanish translation in distribution Hoary.
-
- >>> sourcepackagenameset = getUtility(ISourcePackageNameSet)
- >>> sourcepackagename = sourcepackagenameset['evolution']
- >>> distributionset = getUtility(IDistributionSet)
- >>> distribution = distributionset['ubuntu']
- >>> release = distribution['hoary']
- >>> potemplateset = getUtility(IPOTemplateSet)
- >>> potemplatesubset = potemplateset.getSubset(
- ... distroseries=release, sourcepackagename=sourcepackagename)
- >>> potemplate = potemplatesubset['evolution-2.2']
- >>> pofile_changed = potemplate.getPOFileByLang('es')
-
-This translation file already contains one message which has been changed in
-Ubuntu.
-
- >>> potmsgsets = list(pofile_changed.getPOTMsgSetChangedInUbuntu())
- >>> len(potmsgsets)
- 1
-
- >>> stats = pofile_changed.updateStatistics()
- >>> print pofile_changed.updatesCount()
- 1
-
-This message is also counted as translated message (number of translated
-messages is the number of imported/currentCount plus number of newly
-translated in Ubuntu/rosettaCount plus number of changed in
-Ubuntu/updatesCount):
-
- >>> print pofile_changed.currentCount()
- 6
-
- >>> print pofile_changed.rosettaCount()
- 1
-
- >>> print pofile_changed.updatesCount()
- 1
-
- >>> print pofile_changed.newCount()
- 0
-
-We make sure that submissions on untranslated strings are not considered
-'changed in Ubuntu', since this method is mainly designed to help
-translators revert translations to upstream translations.
-
-Lets get a single untranslated POTMsgSet.
-
- >>> untranslated = list(pofile_changed.getPOTMsgSetUntranslated())[0]
- >>> untranslated is None
- False
-
- >>> current = untranslated.getCurrentTranslationMessage(
- ... pofile_changed.potemplate, pofile_changed.language)
- >>> current is None
- True
-
- >>> imported = untranslated.getImportedTranslationMessage(
- ... pofile_changed.potemplate, pofile_changed.language)
- >>> imported is None
- True
-
-Now, lets provide a translation for it.
-
- >>> translation_message = untranslated.updateTranslation(pofile_changed,
- ... pofile_changed.owner, {0: u'sample translation'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
-
-And make sure that the count of changed in Ubuntu hasn't changed.
-
- >>> potmsgsets = list(pofile_changed.getPOTMsgSetChangedInUbuntu())
- >>> len(potmsgsets)
- 1
-
- >>> stats = pofile_changed.updateStatistics()
- >>> print pofile_changed.updatesCount()
- 1
-
-Now, lets change translation for a message which already has an imported
-translation. First we need to find a message with current, imported
-translation:
-
- >>> for potmsgset in pofile_changed.potemplate:
- ... tm = potmsgset.getImportedTranslationMessage(
- ... pofile_changed.potemplate, pofile_changed.language)
- ... if tm is not None and tm.is_complete and tm.is_current_ubuntu:
- ... imported_potmsgset = potmsgset
- ... break
- >>> potmsgset is None
- False
-
-Lets re-translate that:
-
- >>> translation_message = imported_potmsgset.updateTranslation(
- ... pofile_changed, pofile_changed.owner, {0: u'new translation'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
- >>> from canonical.database.sqlbase import flush_database_caches
- >>> flush_database_caches()
-
-And make sure that the count of messages changed in Ubuntu _has_ changed:
-
- >>> potmsgsets = list(pofile_changed.getPOTMsgSetChangedInUbuntu())
- >>> len(potmsgsets)
- 2
-
-Lets check that the updatesCount() is also properly updated.
-
- >>> stats = pofile_changed.updateStatistics()
- >>> pofile_changed.updatesCount()
- 2
-
-
-getPOTMsgSetWithNewSuggestions
-------------------------------
-
-This method returns only those POTMsgSet's which contain suggestions
-submitted after the last review date, or, in other words, those
-suggestions which need to be reviewed.
-
-Lets get Evolution Spanish translation in distribution Hoary.
-
- >>> person_set = getUtility(IPersonSet)
- >>> potemplateset = getUtility(IPOTemplateSet)
- >>> evolution = getUtility(IProductSet).getByName('evolution')
- >>> evolution_trunk = evolution.getSeries('trunk')
- >>> potemplatesubset = potemplateset.getSubset(
- ... productseries=evolution_trunk)
- >>> evolution_template = potemplatesubset['evolution-2.2']
- >>> evolution_es = evolution_template.getPOFileByLang('es')
-
- >>> noneditor = person_set.getByName('no-priv')
- >>> editor = person_set.getByName('carlos')
-
-Non-editor can only submit a new suggestion on untranslated message.
-
- >>> evolution_es.getPOTMsgSetWithNewSuggestions().count()
- 2
-
- >>> stats = evolution_es.updateStatistics()
- >>> print evolution_es.unreviewedCount()
- 2
-
- >>> potmsgset = evolution_es.potemplate.getPOTMsgSetByMsgIDText(
- ... u'evolution minicard')
- >>> current = potmsgset.getCurrentTranslationMessage(
- ... evolution_es.potemplate, evolution_es.language)
-
- # Make sure this entry is unreviewed and untranslated
-
- >>> print current
- None
-
- >>> new_translation = potmsgset.updateTranslation(
- ... evolution_es, noneditor, {0: u'translation'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
- >>> flush_database_caches() # Store modification dates in database
- >>> print new_translation.translations
- [u'translation']
-
- >>> new_translation.is_current_ubuntu
- False
-
- >>> current = potmsgset.getCurrentTranslationMessage(
- ... evolution_es.potemplate, evolution_es.language)
- >>> print current
- None
-
- >>> evolution_es.getPOTMsgSetWithNewSuggestions().count()
- 3
-
- >>> stats = evolution_es.updateStatistics()
- >>> print evolution_es.unreviewedCount()
- 3
-
-A privileged translator can approve this same suggestion by submitting
-it himself.
-
- >>> new_translation = potmsgset.updateTranslation(
- ... evolution_es, editor, {0: u'translation'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
- >>> flush_database_caches() # Store modification dates in database
- >>> current = potmsgset.getCurrentTranslationMessage(
- ... evolution_es.potemplate, evolution_es.language)
- >>> print current.translations
- [u'translation']
-
- >>> evolution_es.getPOTMsgSetWithNewSuggestions().count()
- 2
-
- >>> stats = evolution_es.updateStatistics()
- >>> print evolution_es.unreviewedCount()
- 2
-
-Or, they can update it with entirely different translation, which won't
-be picked up as a new suggestion, but will be directly set as the
-translation.
-
- >>> new_translation = potmsgset.updateTranslation(
- ... evolution_es, editor, {0: u'boohoo'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
- >>> flush_database_caches() # Store modification dates in database
- >>> current = potmsgset.getCurrentTranslationMessage(
- ... evolution_es.potemplate, evolution_es.language)
- >>> print current.translations
- [u'boohoo']
-
- >>> evolution_es.getPOTMsgSetWithNewSuggestions().count()
- 2
-
- >>> stats = evolution_es.updateStatistics()
- >>> print evolution_es.unreviewedCount()
- 2
-
-Non-editors can still provide suggestions on messages with approved
-translations, and this will again list this message among those with new
-suggestions.
-
- >>> potmsgset = evolution_es.potemplate.getPOTMsgSetByMsgIDText(
- ... u'have ')
- >>> current = potmsgset.getCurrentTranslationMessage(
- ... evolution_es.potemplate, evolution_es.language)
- >>> print current.translations
- [u'tiene ']
-
- >>> new_translation = potmsgset.updateTranslation(
- ... evolution_es, noneditor, {0: u'blahblah'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
- >>> current = potmsgset.getCurrentTranslationMessage(
- ... evolution_es.potemplate, evolution_es.language)
- >>> print current.translations
- [u'tiene ']
-
- >>> evolution_es.getPOTMsgSetWithNewSuggestions().count()
- 3
-
- >>> stats = evolution_es.updateStatistics()
- >>> print evolution_es.unreviewedCount()
- 3
-
-And if we've got two new suggestions for the same message, POTMsgSet
-will not be listed twice in the returned result.
-
- >>> new_translation = potmsgset.updateTranslation(
- ... evolution_es, noneditor, {0: u'another'},
- ... is_current_upstream=False,
- ... lock_timestamp=datetime.datetime.now(UTC))
- >>> evolution_es.getPOTMsgSetWithNewSuggestions().count()
- 3
-
- >>> len(set(evolution_es.getPOTMsgSetWithNewSuggestions()))
- 3
-
-
People who contributed translations
-----------------------------------
@@ -867,6 +591,12 @@
... print person.name
... print "--"
+ >>> evolution = getUtility(IProductSet).getByName('evolution')
+ >>> evolution_trunk = evolution.getSeries('trunk')
+ >>> potemplatesubset = potemplateset.getSubset(
+ ... productseries=evolution_trunk)
+ >>> evolution_template = potemplatesubset['evolution-2.2']
+ >>> evolution_es = evolution_template.getPOFileByLang('es')
>>> print_names(evolution_es.contributors)
carlos
mark
@@ -887,7 +617,6 @@
valyag
name16
name12
- ubuntu-translators
tsukimi
--
@@ -954,7 +683,6 @@
current... 0 carpeta... False
have 0 tiene False
cards 0 tarjetas False
- evoluti... 0 boohoo False
The loc... 0 La ubic... False
%d contact 0 %d cont... True
%d contact 1 %d cont... True
@@ -969,6 +697,7 @@
This method returns a list of TranslationMessages in a given POFile
created by a certain person.
+ >>> person_set = getUtility(IPersonSet)
>>> carlos = person_set.getByName('carlos')
>>> translationmessages = evolution_es.getTranslationsFilteredBy(carlos)
>>> for translationmessage in translationmessages:
@@ -978,10 +707,14 @@
[u'lalalala']
[u'tiene ']
[u' tarjetas']
- [u'boohoo']
- [u'La ubicaci\xf3n y jerarqu\xeda de las carpetas de contactos de Evolution ha cambiado desde Evolution 1.x.\n\nTenga paciencia mientras Evolution migra sus carpetas...']
+ [u'La ubicaci\xf3n y jerarqu\xeda de las carpetas de contactos de
+ Evolution ha cambiado desde Evolution 1.x.\n\nTenga paciencia mientras
+ Evolution migra sus carpetas...']
[u'%d contacto', u'%d contactos']
- [u'Abrir %d contacto abrir\xe1 %d ventanas nuevas tambi\xe9n.\n\xbfQuiere realmente mostrar este contacto?', u'Abrir %d contactos abrir\xe1 %d ventanas nuevas tambi\xe9n.\n\xbfQuiere realmente mostrar todos estos contactos?']
+ [u'Abrir %d contacto abrir\xe1 %d ventanas nuevas tambi\xe9n.\n\xbfQuiere
+ realmente mostrar este contacto?',
+ u'Abrir %d contactos abrir\xe1 %d ventanas nuevas tambi\xe9n.\n\xbfQuiere
+ realmente mostrar todos estos contactos?']
[u'Contrase\xf1a de EncFS: ']
If the passed person is None, the call fails with an assertion.
@@ -1004,7 +737,6 @@
>>> alsa_trunk = alsautils.getSeries('trunk')
>>> alsa_template = alsa_trunk.getPOTemplate('alsa-utils')
>>> alsa_translation = alsa_template.newPOFile('sr')
- >>> alsa_translation.owner = editor
This translation file contains a translation-credits message. By default
it is created with a dummy translation
=== modified file 'lib/lp/translations/tests/test_translations_to_review.py'
--- lib/lp/translations/tests/test_translations_to_review.py 2010-10-18 16:36:46 +0000
+++ lib/lp/translations/tests/test_translations_to_review.py 2010-11-30 16:59:29 +0000
@@ -79,10 +79,10 @@
potemplate=self.potemplate, language_code='nl'))
self.potmsgset = self.factory.makePOTMsgSet(
potemplate=self.potemplate, singular='hi', sequence=1)
- self.translation = self.factory.makeTranslationMessage(
+ self.translation = self.factory.makeCurrentTranslationMessage(
potmsgset=self.potmsgset, pofile=self.pofile,
translator=self.person, translations=['bi'],
- date_updated=self.base_time)
+ date_created=self.base_time, date_reviewed=self.base_time)
later_time = self.base_time + timedelta(0, 3600)
self.suggestion = removeSecurityProxy(
@@ -91,7 +91,6 @@
translator=self.factory.makePerson(), translations=['wi'],
date_created=later_time))
- self.assertTrue(self.translation.is_current_ubuntu)
self.pofile.updateStatistics()
self.assertEqual(self.pofile.unreviewed_count, 1)
Follow ups
