launchpad-dev team mailing list archive

[Brad Crittenden <brad.crittenden@xxxxxxxxxxxxx>]

On Dec 8, 2010, at 10:34 , Aaron Bentley wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 12/08/2010 09:14 AM, Brad Crittenden wrote:
> 
>> "Tests for a class should complete in under 2 seconds. If they aren't, spend at least a little time determining why."
>> 
>> To incorporate this metric into your reviews you'll obviously need to run the tests.  If a test is brand new then you can just evaluate the timing.  Modifications to existing tests should be compared with before and after timings.  So reviewers, either download the code being reviewed and run the tests yourself or ask the developer for comparative timings.  Start the conversation.
> 
> You say that we can incorporate this metric by checking the timing of
> new and modified tests, but the metric is about all tests for a class.
> In order to derive the times for all tests for the class from the timing
> of new and modified tests, we would have to already know the cumulative
> time of tests for the class.  We don't have that, and can't easily
> generate it, because execution times will vary depending on developer
> machines.
> 
> I think you're proposing a new metric instead of incorporating the one
> you quote.  Could you please spell out your new metric, so that I can
> consider it during my reviews?
> 
> Aaron


Hi Aaron,

I tried to explain my intentions during the reviewers meeting last week but clearly failed for you.  After much discussion I thought Julian's summary was good:

[15:19] <bigjools> can we paraphrase this as "please review how long your tests are taking"

As Jono later indicated, we probably don't want to suggest a lot of non-intuitive changes for the sake of speeding up the test as we'll have major changes to our test infrastructure in the near future.  I am asking that developers and reviewers take test timing into consideration.  Easy things such as the example I cited of creating a view directly rather than using the user browser is a big speed up when applicable.

--Brad