launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

On Wed, Aug 11, 2010 at 3:09 AM, Paul Hummer <paul.hummer@xxxxxxxxxxxxx> wrote:
> On Tue, 10 Aug 2010 22:09:56 +1000
> Martin Pool <mbp@xxxxxxxxxxxxx> wrote:
>
>> I'm glad you're doing performance Tuesday.  (I wonder if a one-day
>> timeslice is enough to really enough for the average bear to land
>> anything in lp, but perhaps it is if you're all ready to go.)
>
> I wondered this too, but I've been reading Robert's emails hoping to
> find some tools that could allow us to make small performance changes
> in a day, as well as keep you from rabbit holing.

So a few are starting to crop up - or at least repeated patterns of
issues. I'm seeing:
 - queries that need a bit of manual love to work well
 - death-by-1000-queries

as common repeated patterns.

The former are commonly fixed by asking Stuart/playing on staging and
then generating a trivial patch to fix. You can get Stuarts attention
on bugs by tagging them 'dba' (and please remove the tag when he's
commented so its out of his queue). Playing on staging is harder for
non-team-leads, and there are data confidentiality issues making
changing that a little nontrivial; however we might be able to do
something with explain analyze, though not in the next few weeks :).

The latter are trickier because we want to get all the related data we
want at once, no more than needed, and no less. If we get more things
go slower, and if we get less we scale poorly as the page gets busier
(e.g. more builds). The pattern I've tried in registry with
_all_members should help considerably, once we get the cache coherency
stuff sorted (which strictly speaking only really affects the test
suite : production clears storm caches between requests, and so caches
on model objects have no lifetime in the actual server). I think doing
something similar to that _all_members patch I've done would be a good
experiment and fairly cheap to do.

> I got the feeling from talking with Robert at the Epic that we need to
> have a cultural change where we build tools that will help us solve
> complex problems easier.

Yes! When something is a bit tricky or manual, and you can make it
easier / more automatic - thats a great thing to do.

I'm about to tweak the page performance report, for instance, because
I want to know which pages are the next ones likely to start OOPSing.

And in the longer term, we probably want ORM and transaction framework
help to manage the job of making 'attributes of objects should be
cheap' and 'data we need later should be retrieved with the immediate
content' easy to express and execute upon.

-Rob