ubuntu-translations-coordinators team mailing list archive

[Milo Casagrande <milo@xxxxxxxxxxxxxxx>]

** Changed in: ubuntu-translations
       Status: New => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Translations Coordinators, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/877195

Title:
  Move statistics updating outside the web browser code

Status in Launchpad itself:
  Fix Released
Status in Ubuntu Translations:
  Fix Released

Bug description:
  At the moment, we have a lot of problems with translation statistics
  not being updated real-time even though we make every attempt to do
  so. I am suspecting this fails mostly due to timeouts in POSTs when we
  issue POFile.updateStatistics() call.

  We have so far solved this with band-aids like cronscripts/rosetta-
  pofile-stats.py and cronscripts/rosetta-pofile-stats-daily.py, which
  have their own problems (bug 622668 and bug 781274). These scripts
  became even more important since message-sharing was introduced,
  because POFile.updateStatistics() updated the statistics only on a
  single POFile that was being touched, even though changes on its
  messages affect "sharing" POFiles as well. pofile-stats-daily script
  specifically has logic to find these "sharing" files, though it might
  be incomplete.

  Instead of trying to fix our band-aids, we should fix the root cause
  instead. We know that sometimes it's going to be impossible to both
  save all the changes on a single +translate page *and* calculate the
  statistics (eg. ddtp-ubuntu templates are over 40k messages total) in
  a single POST, and we also know that statistics being very up-to-date
  real-time is not essential (we have gotten translators used to them
  being a week or so behind; having them eg. 5 minute behind at *most*
  would be a huge improvement).

  So, we should instead decouple translation saving from statistics update. I propose we move any updateStatistics calls from the browser code (though, there should be only one) to a job which is scheduled to run whenever there are changes to a POFile. We should also extend updateStatistics call to support finding all the sharing POFiles as well (all the potentially touched POFiles).
  The same should be done for POTemplate.newPOFile callsite which is called from within the browser code as well.

  Other updateStatistics callsites (eg. in model/pofile.py and another
  one in model/potemplate.py importFromQueue) are related to imports
  which are already de-coupled from the browser code, so can (and
  should) stay "inline". Having them fix the statistics for all the
  sharing POFiles would be nothing but an improvement.

  Once all of the above is done, we should be able to kill rosetta-
  pofile-stats-daily.py script, and leave rosetta-pofile-stats.py script
  around just in case we want to run it manually (we should remove it
  from the crontab or at least make is less common). Since it would go
  over all the POFiles, we should ensure that one doesn't try to find
  all the sharing POFiles when it's calling updateStatistics on one
  POFile.

  Fixing this bug would solve bug 781274 (a critical regression) and bug
  622668, thus marking this one as critical and adding notes there.

To manage notifications about this bug go to:
https://bugs.launchpad.net/launchpad/+bug/877195/+subscriptions