launchpad-dev team mailing list archive

[Robert Collins <robert.collins@xxxxxxxxxxxxx>]

On Wed, Aug 11, 2010 at 10:44 AM, Edwin Grubbs
<edwin.grubbs@xxxxxxxxxxxxx> wrote:
> On Tue, Aug 10, 2010 at 5:20 PM, Robert Collins
...
>> Makes sense to me at a casual glance; this is orthogonal to /when/ we
>> do the work though, IMO.
>
>
> The reason I think we shouldn't batch filling in the TeamParticipation
> table unless absolutely necessary is that the members of the newly
> added team don't get any of the benefits until that is done. We also
> have to provide some kind of status indicator and possibly an
> estimated time till completion. If a specific person needs the
> superteam's privileges right away, the superteam's admin may feel
> forced into adding that person directly instead of waiting for the
> queue to process the team.
>
> -Edwin

Batching to me is not the same as queuing and perhaps we are crossing
wires here?

I think you are saying 'the user wants their action to happen now, not
in N minutes'.

I agree, they do - and putting the work through a queue neither helps
nor hinders them directly.

Here's what an ideal experience would look like from the users perspective:
 - they click ok
 - the web page shows a spinner for a second or so
 - they get a confirmation message

>From our end, I'd love that to look like this:
 - we receive a request
 - we commit the *intent*, dispatch to a queue
 - the request puts itself into long-poll mode
 - the queue processes the work
 - the queue callsback the appserver which reactivates the request
which then delivers the users response

Thats going to need some serious plumbing, a rearrangement of /all/
the global-request-specific-state and so on. Not on the cards for now.

What we can do now is:
 - we receive the request
 - we commit the *intent* and return a 'pending' message to the
browser, which then renders a spinner locally and polls an API for
completion of the queued item
 - the queue processes the work
 - the polling js sees the work is complete and renders completion

Now, queue latency and when the user gets their benefits. The current
job system has on average 30 second latency + processing time. We can
address that for specific queues with a small amount of work to poll
the DB every second in a new transaction.

This should be pretty transparent to the user, will make the page time
come -way- down, and handle very large team rearrangements gracefully.

-Rob