launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

So this has been discussed a bit in a couple of threads; and I wanted
to bring it all together.

Firstly, we already cache, in an adhoc manner, in model objects. (Case
in point: shipit has code to clear(erroneously as it happens) a cache
in Person.

Secondly, we don't have any cache invalidation patterns at the moment,
other than some adhoc and all-slightly-different ones (some things use
storm hooks, others explicit methods...).

So, lp:~lifeless/launchpad/cachedproperty is a branch which cleans
this up : it provides an API for working in more detail with cached
properties; it makes SQLBase automatically invalidate cachedproperty
caches, and it provides a way for things that know a cache needs to be
updated, or removed, to do so across the system. This branch is in ec2
land mode at the moment.

I will update the storm notes on the wiki, but here is key message:
In SQLBase classes, do not write __storm_invalidation__ hooks anymore:
just use cachedproperty, and its all taken care of for you. If you
*do* write a hook, be sure to upcall, or else cachedproperty caches
won't be cleared.

If you are writing pure storm without SQLBase, either don't use
cachedproperty at all, or call clear_cachedproperties(self) in a storm
invalidation hook.

Thirdly, I have a branch which adds moderately extensive caching to
Person, with an optional helper to get all the members of a team with
all their caches preloaded, this makes /participation on a team take
11 DB queries (and 4 of those are OAuth). I think its a reasonable
approach to the short term challenge of getting a bunch of related
data for a page or API, and I'm interested in how easy folk find it to
read or reuse. There is more we can do to make the idiom work well and
be reusable, but I think its pretty good for a early step. (Its very
similar to the sorts of methods Danilo talked about, but nicer to use
as a consumer, I think). This branch is also reviewed and should be in
ec2 land anytime now. It also adds some polish to the cachedproperty
helper functions, because until you use something in anger its hard to
be sure its complete (it wasn't).

When caching, please do so with some thought to likely actions that
will need to clear the cache or otherwise modify it.

Cheers,
Rob