subunit-dev team mailing list archive

[Jonathan Lange <jml@xxxxxxxxx>]

On Sun, Jul 19, 2009 at 8:45 AM, Robert
Collins<robertc@xxxxxxxxxxxxxxxxx> wrote:
> This is a bit of a brain dump. I had a brief chat with jml tonight about
> what sort of timing-data reports would be good to get out about test
> runs, and I've been thinking since about how they interact with subunit
> and so on.
>
> Firstly, some things it would be rad to get good data on:
> slow <x> where <x> is:
> 'setup' methods, 'teardown' methods, 'total test time', 'resource
> creation time', 'resource deletion time'
>
> then perhaps the first N for those things, perhaps cross reference by
> tags, or regexes on testids etc.
>
> Being more granular than 'total test time' assumes that more granularity
> is helpful. I'm not entirely sure it is - after all, all tests in the
> same class will likely have the same time for setup,so all the tests
> will show up as slow. OTOH things like testresources have interesting
> data about *when* resource rebuilding takes place that we would like to
> capture, and knowing about duration doing that would be good.
>
>
> We have a time tag in subunit already; I'm going to use that to get
> basic python test timing data recorded. That won't be enough to get
> testresources activity recorded, nor setup/teardown time, or other
> things people may want.
>
> So I'm wondering how best to extend this. We could:
> - add an arbitrary event command:
> event TAG
> test foo
> time 1
> event setup-start
> time 2
> event setup-stop
> time 3
> event teardown-start
> time 4
> event teardown-stop
> time 5
> fail foo [
>  exception blah
> ]
>
>
> - also optionally time stamp every command [I don't really care for this
> but perhaps it would be cleaner]
>
> But I'm seeking other ideas!
>

Hey Rob,

It's been quite some time since you sent this email out, and you seem
to have done a lot of timing-related work to bzr. Did you come to any
conclusions about this design?

jml