launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Sat, Mar 26, 2011 at 9:04 AM, Benji York <benji.york@xxxxxxxxxxxxx> wrote:
> On Thu, Mar 24, 2011 at 10:11 PM, Robert Collins
> <robert.collins@xxxxxxxxxxxxx> wrote:
>> This is about optimising a particular part of the current webservice;
>> its still useful in an expand/filter world I think, because some
>> collections are massive.
>
> If I'm understanding the various moving parts correctly, the
> filter/expand mechanism would remove the need for this optimization.
> Not that we shouldn't do it for the short term gains (about which I have
> no opinion one way or the other).
>
> How would filter/expand help in this scenario?  It will first return the
> membership of whatever set you specify and the client would then batch
> using that information.  There isn't any need to do batch calculations
> on the server side, avoiding the expensive surrogate key calculations.

the filter/expand system still runs into trouble in very large
batches; we don't know how large 'very large' is because we haven't
written it yet. A trivial example of where very large is too large:
ubuntu/allbugsever

So, making batching /possible/ with very high offsets is necessary at
some point [or we have to stop offering stupendously large collections
in the way we do today]

-Rob