launchpad-dev team mailing list archive

[John Arbash Meinel <john@xxxxxxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/28/2011 05:03 PM, Martin Pool wrote:
> I think for _cold_ cold you ought to drop buffers first, in which case
> I really doubt it can be 31ms, though it might still be under 1000ms.
> Or perhaps a more realistic case is that all the bzr code is in cache,
> but none of the actual branch data.  And it will have to be a (fast)
> magnetic disk, not SSD.

Right, so I expect that bzrlib is going to fully paged into memory. The
issue is that the disk files (.bzr/*) is hard-io.

> 
> Most of what goes into lp appserver pages will come from ram, so
> anything that causes hard IO, as the codehost may, needs care.

Right. And the fact that Loggerhead is actually accessing the Branch
objects over HTTP, and a bunch of other semi-artificial overheads forced
on Loggerhead. (The branch name => disk path abstraction requires an
XMLRPC lookup, which I've seen take 100ms by itself.)

> 
> I hope that we can eventually get there though, at least to the point
> of reading limited-size objects like revisions, but perhaps this will
> only happen when lp has moved to something that allows scatter/gather
> template filling.
> 
> Martin

Sure. My specific point was to qualify the "getting diff is fast". We
can be pretty darn fast in the right circumstances, and the size of the
project should only effect this logarithmically. (btree indexes are
~log100 scaling, chk maps scale ~logarithmically, though they can blow
out if you have >100 file changes in a given commit, since it ends up
changing most pages because of the hash layout.)

I won't say that these are representative of production loggerhead
values. But it does validate my "bzrlib's scaling is appropriate, even
at web scale".

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4J8HIACgkQJdeBCYSNAAMpLwCfalKen8c0MZeg1wjD9weDB6Im
IIkAoKe+tx3+8NWwFZ+hT+xvr3/LaT6W
=QZx5
-----END PGP SIGNATURE-----