← Back to team overview

launchpad-dev team mailing list archive

Re: performance tuesday - changing timeouts, new server progress

 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 04/19/2011 06:01 AM, Robert Collins wrote:
> Some -very- good news.
> 
> With the combination of performance work and the reconfigured
> appservers we've now reduced our 99th percentile to 1.59 seconds
> (measured on sunday). When I started as TA 10 months back our 99th
> percentile was 2.24 seconds. This is slow and tedious work, but
> immensely valuable to our users. I'm really happy with the progress
> we've made - we're 50% of the way to having a 1 second 99th
> percentile.

This is really great news. I'm curious how this works with your in-DC
queuing? At least, from what I remember in the past, your stats did not
take that into account.

It is really nice to see that both the high-end (timeout reduced to 10s)
and the low end (99th percentile at 1.59s) is getting better. It will
certainly be interesting to see if/when you switch from "how can we make
this terrible query less than 10s" to "how can we make these fast 2s
queries faster (<1s)".

...

> 
> We're down to 316 timeouts - 0.004% of requests, so, with an exception
> for Question:+index (which could spike by 200 timeouts a day) the hard
> timeout has been dropped to 10 seconds.
> 
> If it looks good - and it may - we'll drop to 9 seconds later this
> week; meeting Francis challenge for the Epic and leaving only 4
> seconds to go to reach my long term timeout of 5 seconds.
> 
> Of course, we're going to need better and better code and schemas to
> continue improving things, but we're in pretty good shape!
> 
> -Rob

Do we know how many exceptions we have at 10s, and how many exceptions
we would need at 5s?

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk2tWVYACgkQJdeBCYSNAAMnEwCfaOgAYQT3WvSSjieP77t3Q2YC
lOsAoK/EXB17+z6ihJ8DTdGfJ0fAw7Y6
=hbHt
-----END PGP SIGNATURE-----


Follow ups

References