launchpad-dev team mailing list archive

[Danilo Šegan <danilo@xxxxxxxxxxxxx>]

У чет, 26. 08 2010. у 02:07 +0700, Jeroen Vermeulen пише:
> On 2010-08-25 18:32, Danilo Šegan wrote:
> 
> > So, it's likely poimport that does a lot more writes (including writes
> > to pofiletranslator).  However, it's also does a single file import in a
> > single transaction, so it shouldn't be too many changes.  We do have
> > some files which are huge, though (ddtp-ubuntu have around 40k messages:
> > everything else is less than 10k messages, with only a small number of
> > them not less than 2k).
> 
> We're also duplicating each POFileTranslator update across all sharing 
> POFiles nowadays.  Which means that a single TranslationMessage update 
> can be multiplied by the number of templates that share it.  I think for 
> Ubuntu right now, that can be as many as 8.
> 
> 
> > Note that we currently get spikes of how many files we've got to do due
> > to Ubuntu[1].  The nature of our implementation makes it not run
> > as-fast-as-possible (script run is limited to 9 mins, with a pause
> > between runs).  Also, it's already heavily optimized to do as little
> > writes as possible on translationmessage table, but not on the
> > pofiletranslator table.  pofiletranslator is maintained by a trigger
> > (which is overly complex and could probably be optimized).
> 
> Say...  I think we should do that, but until then, what about this 
> short-term fix?

When I say "optimized" for a trigger, I usually mean "removed" :)

> All the POFileTranslator records that the trigger inserts/updates during 
> an import should be identical except in which TranslationMessage they 
> refer to.  And which TranslationMessage they refer to is actually pretty 
> arbitrary--AFAIC it doesn't _have_ to be the last updated one in the 
> file.  It could just as validly be the first updated one in the file.

I think it would be just as simple to recalculate all the translators
along with a statistics update.  And "pofile.updateTranslators" call
should be separate anyway (so we could do it in the same manner we do
updateStatistics today: once per web page submission, or once per
import).

Still, I'd prefer to get updateStatistics (and updateTranslators) call
out of the webapp code first.

Cheers,
Danilo