launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

So as you probably know, bugsummary turned out to be an alias for
'massive eager locking win!'

Stuart put together an interation on the design to address the locking
issues (which went live on Friday last week) and is in devel since
Tuesday. He now has a patch now in db-devel to extend bugsummary to
let us do the bug count portlets.

Journals to avoid contention
=====================

So the basic change we made was from having one table which had too
much contention on single rows (even after 2 rounds of optimisation to
reduce unneeded writes) to having each transaction insert into a
journal the changes it wanted made to the master bugsummary table. The
journal is applied very now and then from cron. Appservers query both
the journal and the bugsummary, combining the results - so the data is
live, but there is only one process writing to the summary table.

Bug count portlets
=============

We have some hairy queries at the moment to determine 'bugs with fixes
upstream' and the like - the output of those checks has been
denormalised into bugsummary with Stuarts latest patch; This increases
the row count consulted to do tags portlet calculations by 50% (12K
rows for Ubuntu), but simultaneously will let us avoid table scans of
bug and bugtask to do the portlets. \o/.

-Rob