launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

Sadly, I didn't end up working directly on a performance issue
yesterday; however I did get (much) of the way through increasing the
transparency on some OOPS.

I started out looking at
https://lp-oops.canonical.com/oops.py/?oopsid=1695L1219 which is
https://bugs.edge.launchpad.net/malone/+bug/438116.

The oops summary is this:
SQL time: 1871 ms
Non-sql time: 43355 ms
Total time: 45226 ms
Statement Count: 490
which, while it has a high query count, is a tolerable amount of sql
time compared to the non-sql time. But what is that nonsql time?

Theres a 40 second gap between a fairly innocuous call:
486.	5408	4ms	launchpad-main-master	UPDATE Bug SET
heat=calculate_bug_heat(622129), heat_last_updated=CURRENT_TIMESTAMP
AT TIME ZONE 'UTC' WHERE Bug.id = %s
and the end of request glue:
487.	45213	2ms	session	UPDATE SessionData SET last_accessed =
CURRENT_TIMESTAMP WHERE client_id = %s AND last_accessed <
CURRENT_TIMESTAMP - '540 seconds'::interval

This action sends email, so its a pretty good bet that that is it. But
- how many emails is it sending? Aren't emails from bugs meant to be
deferred *anyway* ?

So I've dug into webapp.adapter and excised its treatment of
'request_statements' - instead there is now
'lp.services.timeline.requesttimeline' which can be used for anything
that is going to be doing IO or other blocking activities. Its thunked
across into the existing oops reports (which will be a little
confusing but tolerably I think) until we tweak the oops tools to show
them differently.

I have, however, run into some nasty cruft where the webapp 'adapter'
module has a very different idea about 'request is active' to what
zope does. Zope looks at participations; the adapter module looks at
thread local. This is kindof the very thing I'm worried about with
eventually adding in AJAX and so on - so I'm sttuborning it out and
trying to keep the new code very clean and decoupled; I may give up
soon and collapse the tracking component back into the core. I've
filed http:/pad.lv/623199 about this - its may be  a dupe, but there
are workaround comments pointing this out without bug references (like
lib/canonical/launchpad/scripts/__init__.py line 115) so I'm hopeful
that it isn't. Fixing 623199 will probably let us make a lot of code -
the launchpad, the webapp adapter, errorlogging and so on a lot
cleaner.

-Rob