launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Wed, Apr 6, 2011 at 7:16 AM, Robert Collins
<robertc@xxxxxxxxxxxxxxxxx> wrote:
> On Tue, Apr 5, 2011 at 11:49 PM, John Arbash Meinel
> <john@xxxxxxxxxxxxxxxxx> wrote:
>> I understood the earlier discussion about the logic/analysis behind the
>> change. I was wondering if you had actual hard data, since you've done
>> the implementing.
>
> Oh, no not yet: I need to finish this layer change off, then migrate a
> collection in Launchpad itself to provide an appropriate RangeFactory,
> land them all and we can see how it behaves on qastaging.
>
> I could manually test on a given collection, but haven't felt the urge ;)

I now have some interim stats on a particular slow-batch.

This is the bug:
https://bugs.launchpad.net/launchpad/+bug/730396
Getting a specific batch out (hot cache) on qa staging with the
current infrastructure is ~600ms.
Getting the same batch out using a date constraint is 20ms.

Getting the last batch out is 2200ms hot in both cases, because the
table structure requires a tablescan to detect end of batch. (Which is
an issue in its own right. Fixing that will make getting the batches
faster in the current infrastructure too, as well as help in getting
batch size fast).

-Rob