launchpad-dev team mailing list archive

[Edwin Grubbs <edwin.grubbs@xxxxxxxxxxxxx>]

On Mon, Aug 16, 2010 at 6:11 AM, Robert Collins
<robert.collins@xxxxxxxxxxxxx> wrote:
> So this has been discussed a bit in a couple of threads; and I wanted
> to bring it all together.
>
> Firstly, we already cache, in an adhoc manner, in model objects. (Case
> in point: shipit has code to clear(erroneously as it happens) a cache
> in Person.
>
> Secondly, we don't have any cache invalidation patterns at the moment,
> other than some adhoc and all-slightly-different ones (some things use
> storm hooks, others explicit methods...).
>
> So, lp:~lifeless/launchpad/cachedproperty is a branch which cleans
> this up : it provides an API for working in more detail with cached
> properties; it makes SQLBase automatically invalidate cachedproperty
> caches, and it provides a way for things that know a cache needs to be
> updated, or removed, to do so across the system. This branch is in ec2
> land mode at the moment.
>
> I will update the storm notes on the wiki, but here is key message:
> In SQLBase classes, do not write __storm_invalidation__ hooks anymore:
> just use cachedproperty, and its all taken care of for you. If you
> *do* write a hook, be sure to upcall, or else cachedproperty caches
> won't be cleared.
>
> If you are writing pure storm without SQLBase, either don't use
> cachedproperty at all, or call clear_cachedproperties(self) in a storm
> invalidation hook.
>
> Thirdly, I have a branch which adds moderately extensive caching to
> Person, with an optional helper to get all the members of a team with
> all their caches preloaded, this makes /participation on a team take
> 11 DB queries (and 4 of those are OAuth). I think its a reasonable
> approach to the short term challenge of getting a bunch of related
> data for a page or API, and I'm interested in how easy folk find it to
> read or reuse. There is more we can do to make the idiom work well and
> be reusable, but I think its pretty good for a early step. (Its very
> similar to the sorts of methods Danilo talked about, but nicer to use
> as a consumer, I think). This branch is also reviewed and should be in
> ec2 land anytime now. It also adds some polish to the cachedproperty
> helper functions, because until you use something in anger its hard to
> be sure its complete (it wasn't).
>
> When caching, please do so with some thought to likely actions that
> will need to clear the cache or otherwise modify it.
>
> Cheers,
> Rob

If each request knows whether it needs a read-write or read-only
database connection so that it can connect to the slave or master,
couldn't we add the ability for SQLBase to cache all the attributes if
the request is read-only?

-Edwin