subunit-dev team mailing list archive

[Robert Collins <robertc@xxxxxxxxxxxxxxxxx>]

This is a bit of a brain dump. I had a brief chat with jml tonight about
what sort of timing-data reports would be good to get out about test
runs, and I've been thinking since about how they interact with subunit
and so on.

Firstly, some things it would be rad to get good data on:
slow <x> where <x> is:
'setup' methods, 'teardown' methods, 'total test time', 'resource
creation time', 'resource deletion time'

then perhaps the first N for those things, perhaps cross reference by
tags, or regexes on testids etc.

Being more granular than 'total test time' assumes that more granularity
is helpful. I'm not entirely sure it is - after all, all tests in the
same class will likely have the same time for setup,so all the tests
will show up as slow. OTOH things like testresources have interesting
data about *when* resource rebuilding takes place that we would like to
capture, and knowing about duration doing that would be good.


We have a time tag in subunit already; I'm going to use that to get
basic python test timing data recorded. That won't be enough to get
testresources activity recorded, nor setup/teardown time, or other
things people may want.

So I'm wondering how best to extend this. We could:
- add an arbitrary event command:
event TAG
test foo
time 1
event setup-start
time 2
event setup-stop
time 3
event teardown-start
time 4
event teardown-stop
time 5
fail foo [
  exception blah
]


- also optionally time stamp every command [I don't really care for this
but perhaps it would be cleaner]

But I'm seeking other ideas!

-Rob

Attachment: signature.asc
Description: This is a digitally signed message part