← Back to team overview

launchpad-dev team mailing list archive

Re: Peformance tuesday: batching time!

 

On 2010-09-14 16:44, Robert Collins wrote:

Wrap the resultset with BatchNavigator.

Something to pay attention to here: make sure that you wrap a resultset of some kind (not a list of some kind) and that it has not issued to the database yet.

We've had a few performance bugs where we unknowingly or unthinkingly forced the ORM to issue a query first, and then asked the batch navigator to slice the result. Costs the same as unbatched operation, except the user also needs to repeat the request for every batch.

I've requested a Storm feature to let us check that a result set is not issued before it is sliced.


The new request timeline code was breaking the oops generation code when:
  - the timeout was when a sql request *was not emitted*
  - the oops code tried to send a sql statement itself.

This is now fixed in prod via a cowboy and will be fixed on edge when
Jeroen's fix for the Python2.6 issue lands and gets deployed.

It has been deployed.


Jeroen



References