launchpad-dev team mailing list archive

[Benji York <benji.york@xxxxxxxxxxxxx>]

On Thu, Nov 18, 2010 at 3:59 PM, Gavin Panella
<gavin.panella@xxxxxxxxxxxxx> wrote:
> On 18 November 2010 16:42, Jeroen Vermeulen <jtv@xxxxxxxxxxxxx> wrote:
> ...
>>  * The cost sink in WADL generation is almost half a million calls to
>> add_option in optparse.
>
> Wha?!? How on earth does that make sense? Granted, I'm not trying very
> hard, but that smells really fishy.

Don't try too hard; it really is making all those calls.

While doing the investigation Jeroen mentioned earlier, I found that
while generating HTML from docstrings epydoc was constructing an
instance of a particular class per docstring.  That class invoked
optparse.  I reworked our code to reuse a single instance instead which
reduced the number of function calls while generating the WADL by over
20 million (down from 44 million).

The lazr.restful branch with this optimization in it has been approved
and I'll do a release tomorrow.

I also have a small Launchpad branch that does the WADL generation in
parallel subprocesses which speeds things up as well.

The two branches together reduce the wall time to generate the WADL
(and WADL-based HTML API docs) from 74.7s to 44.03s (a reduction of
just over 40%).
-- 
Benji York