launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Tue, Feb 15, 2011 at 10:17 PM, Martin Pool <mbp@xxxxxxxxxxxxx> wrote:
> One interesting thing I discovered today, while working on
> https://launchpad.net/kanban/, is that the overall time to draw the
> kanban is quite consistently like 30% greater (slower) talking to
> lpnet than to edge.  So, the sooner the edge hardware can be folded
> into lpnet, the better.

The reason for this is different queues in the datacentre: edge has
between 6 and 8 requests active on each appserver at once; lpnet has
7-10.

But both appservers are only configured to run 4 requests
concurrently: those additional requests queue.

This is likely self limiting at the moment: the user population
doesn't push new requests until their prior one completes.

We have 8 cores we could reconfigure too as soon as the losas have
some time; behind that we have 12 additional cores coming in new
machines; these should let us support the active users more directly,
though it may not be sufficient - we'll see how we go.

We're also reconfiguring to make the queuing fairer: rather than
queuing on each appserver we're going to queue in haproxy, so requests
will go to the first available service point, rather than potentially
getting stuck behind 4 bug searches on one appserver. This should
eliminate a lot of the sporadic nature of the delays (but won't
intrinsically remove them).

-Rob