launchpad-dev team mailing list archive

[Gavin Panella <gavin.panella@xxxxxxxxxxxxx>]

Cursor arraysize is the number of rows that psycopg2 fetches each time
fetchmany() is called. Storm's Result.__iter__() rests upon that, and
ResultSet rests upon that, so arraysize ought to bear on performance.

check-teamparticipation.py now slurps lots from the database, and one
thing I tweaked to make it quicker is to hack the cursor arraysize to
be much larger.

I didn't take any measurements specifically about this because I was
iterating other things too, but it appeared to have made slurping
quicker. I'm kicking myself for not having taken proper isolated
measurements, but they wouldn't be too hard to do now.

Perhaps unbelievably, psycopg2 sets arraysize to 1. Storm attempts to
salvage things by setting it to 10 (unless it has already been changed
to something other than 1).

I wonder if we could help Launchpad by setting it to, say, 100, or
even just 20 as a first step?

Unless someone can say it's pointless I'll try and get some
measurements about the performance impact.

Gavin.