launchpad-dev team mailing list archive

[Julian Edwards <julian.edwards@xxxxxxxxxxxxx>]

On Monday 16 August 2010 19:28:21 Jeroen Vermeulen wrote:
> On 2010-08-17 01:24, Robert Collins wrote:
> > Firstly is that caching is only a small benefit itself: it saves us
> > doing precisely identical work for the life of the cache. However
> > caching in the model layer is a prerequisite to *prepopulating the
> > caches*, which is where we get the major wins: that lets us eliminate
> > repeated similar work by doing one query for a set of data rather than
> > one query per related thing *per row*. So the gain of caching
> > everything vs caching what we've figured out is safe is relatively
> > small, because the incremental gain from both of these things is small
> > without other work, and that other work will apply to read only and
> > readwrite transactions.
> 
> It may help to distinguish between prefetching, batch-fetching, and
> caching here.  The caching is instrumental, but not the whole story.

Indeed, I think most of the Soyuz SQL performance fixes involved pre-fetching 
(or prejoins in SQLObject parlance) and batch-fetching.  Michael's 
DecoratedResultSet is instrumental in that strategy.