ubuntu-translations-coordinators team mailing list archive

[Launchpad Bug Tracker <877195@xxxxxxxxxxxxxxxxxx>]

You have been subscribed to a public bug by David Planella (dpm):

At the moment, we have a lot of problems with translation statistics not
being updated real-time even though we make every attempt to do so. I am
suspecting this fails mostly due to timeouts in POSTs when we issue
POFile.updateStatistics() call.

We have so far solved this with band-aids like cronscripts/rosetta-
pofile-stats.py and cronscripts/rosetta-pofile-stats-daily.py, which
have their own problems (bug 622668 and bug 781274). These scripts
became even more important since message-sharing was introduced, because
POFile.updateStatistics() updated the statistics only on a single POFile
that was being touched, even though changes on its messages affect
"sharing" POFiles as well. pofile-stats-daily script specifically has
logic to find these "sharing" files, though it might be incomplete.

Instead of trying to fix our band-aids, we should fix the root cause
instead. We know that sometimes it's going to be impossible to both save
all the changes on a single +translate page *and* calculate the
statistics (eg. ddtp-ubuntu templates are over 40k messages total) in a
single POST, and we also know that statistics being very up-to-date
real-time is not essential (we have gotten translators used to them
being a week or so behind; having them eg. 5 minute behind at *most*
would be a huge improvement).

So, we should instead decouple translation saving from statistics update. I propose we move any updateStatistics calls from the browser code (though, there should be only one) to a job which is scheduled to run whenever there are changes to a POFile. We should also extend updateStatistics call to support finding all the sharing POFiles as well (all the potentially touched POFiles).
The same should be done for POTemplate.newPOFile callsite which is called from within the browser code as well.

Other updateStatistics callsites (eg. in model/pofile.py and another one
in model/potemplate.py importFromQueue) are related to imports which are
already de-coupled from the browser code, so can (and should) stay
"inline". Having them fix the statistics for all the sharing POFiles
would be nothing but an improvement.

Once all of the above is done, we should be able to kill rosetta-pofile-
stats-daily.py script, and leave rosetta-pofile-stats.py script around
just in case we want to run it manually (we should remove it from the
crontab or at least make is less common). Since it would go over all the
POFiles, we should ensure that one doesn't try to find all the sharing
POFiles when it's calling updateStatistics on one POFile.

Fixing this bug would solve bug 781274 (a critical regression) and bug
622668, thus marking this one as critical and adding notes there.

** Affects: launchpad
     Importance: Critical
     Assignee: Benji York (benji)
         Status: Fix Committed

** Affects: ubuntu-translations
     Importance: Undecided
         Status: New


** Tags: qa-needstesting translations-handoff
-- 
Move statistics updating outside the web browser code
https://bugs.launchpad.net/bugs/877195
You received this bug notification because you are a member of Ubuntu Translations Coordinators, which is subscribed to the bug report.