launchpad-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Tue, Jun 28, 2011 at 4:42 AM, Martin Pool <mbp@xxxxxxxxxxxxx> wrote:
> We're working on <https://dev.launchpad.net/Projects/LiveBranches>
> which is to try to get branch content a bit better integrated with the
> main Launchpad UI, along the lines of
> <https://dev.launchpad.net/ArchitectureGuide/ServicesRoadmap#loggerhead+(bazaar.launchpad.net+web+UI)>
>
> We propose to do this by letting client javascript send requests that
> get proxied through the front-end Apache to loggerhead, and then
> changing Loggerhead to serve more machine-readable data: plain diffs
> or file content, or json for things like file and revision metadata.
> Obviously this needs a little care against xss.
>
> Basically we would look at putting something like
> <http://bazaar.launchpad.net/~jelmer/launchpad/apache-config-loggerhead/revision/13262>
> into the front end Apache servers on banana and nutmeg.  mthaddon
> thinks this is reasonable.  Probably it should be more restrictive
> about what URLs are proxied.
>
> If this approach is ok with Robert, I'd ask him to file an rt asking
> for similar rewrite rules to be added, and (if necessary) firewall
> rules to be changed to let those machines get to the loggerhead
> server/s.  We can safely do this ahead of actually deploying code that
> uses them.  We need to also block those URLs from robots.

I've advocated this exact approach; its fine. I think you should file
an RT yourselves with the exact stuff you want done; Francis can
prioritise that for you on the spot.

> Another separate but related thing would be for the web app front end
> to do internal http requests to loggerhead, to get the same kind of
> information, but to render it on the server side.  That probably means
> giving Launchpad a concept for making an outgoing http request within
> the dc, subject to a timeout, with some kind of tracking of when they
> fail.  That seems generally useful in terms of the SOA plan, but
> perhaps harder than doing it on the client, and maybe more likely to
> make the page time out on the server side.  It would mean the page
> will still work for robots and non-js browsers.   However, it would
> mean that robots walking those pages of the main site will create load
> on loggerhead, which could be substantial.

I have serious doubts that loggerhead and bzr are ready for loggerhead
acting as an internal use service. Robots we can deal with (nofollow
attributes on links). 15 second first-time-loads on Launchpad branches
we cannot. Loggerhead is not yet monitored and reported on to the
degree that the main Launchpad appserver is - and until it is its very
hard to be confident that it responds sufficiently *faster* than LP
itself to not cause timeouts in LP.

tl;dr:
+1 on browser using json through apache on the main launchpad domains
(e.g. https://launchpad.net) and having apache proxypass those
specific requests to loggerhead.

-0 on the zope appserver stack requesting json from loggerhead inside
the page render process [a +1 is subject to either a parallelising
render (which is many months away) or loggerhead having a 99th
percentile under 1 second, with none over 5 seconds].

-Rob