launchpad-reviewers team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

On Sat, Sep 11, 2010 at 10:37 PM, Jeroen T. Vermeulen <jtv@xxxxxxxxxxxxx> wrote:
>> This isn't a review, but I'm curious how you've established that this
>> is faster
>
> I haven't actually established that it is faster; I would like to do that based on timings on actual production data on one of the test servers.  In practice it's not so much the speed of producing these listings that we care about, as it is the speed of producing specifically the Ubuntu listings.

Staging is currently not suitable for perf testing except under quite
controlled circumstances (see my bug on foundations - 'staging gets
overloaded') - just a heads up. If I was doing this I think I'd
compare the page before and after locally using the comment field to
get local timings.


>> Also, we have a new template engine in the
>> pipeline :- we probably don't want a lot of procedural code doing
>> template-work (or we want all our code doing that).
>
> This is a bit of a special case given the enormous amounts of data that show up in the page.  (There are good reasons for doing it this way, so "just batch it" is a poor plan B).
>
> If the new template engine turns out to be faster than straight-through python code, then we should probably scuttle this branch.

I don't really like batching most of the time either. As for template
being faster - I wouldn't necessarily expect that, but the tradeoff
between understandability and performance might be less skewed with
chameleon. You could try it - its in the tree now.


>> This isn't as efficient as a pure chunking end-to-end implementation,
>> but it is the next best thing.
>
> I do that for smaller chunks.  For the page as a whole however I think the templating engine is likely to be better tuned.  With literally thousands of lines on a page, the ability to start sending HTML before the full list has been rendered could be a real plus.

With our current design we simply cannot do that:
 - any exception before publication finishes generates an OOPS
 - so no data is sent until we know we won't have an OOPS

-> Don't worry about incremental performance here, worry about total
performance.

-Rob
-- 
https://code.launchpad.net/~jtv/launchpad/templates-listing/+merge/35185
Your team Launchpad code reviewers is requested to review the proposed merge of lp:~jtv/launchpad/templates-listing into lp:launchpad/devel.