testtools-dev team mailing list archive

["Martin \[gz\]" <gzlist@xxxxxxxxxxxxxx>]

However bzr also needs to start deferring to testtools for the timings
in order to pick up the funky way subunit timestamps are handled.

** Changed in: bzr
   Importance: Undecided => Medium

** Changed in: bzr
     Assignee: (unassigned) => Martin [gz] (gz)

-- 
selftest --parallel test case timings incorrect
https://bugs.launchpad.net/bugs/625594
You received this bug notification because you are a member of testtools
developers, which is subscribed to testtools.

Status in Bazaar Version Control System: Confirmed
Status in testtools: Confirmed

Bug description:
Individual test timings appear to be completely off when selftest is run across multiple processes.

Running two reasonably slow tests:

    blackbox.test_log.TestLogTags.test_log_with_tags                                                         OK      483ms
    blackbox.test_log.TestLogTags.test_merged_log_with_tags                                                  OK      468ms
    ----------------------------------------------------------------------
    Ran 2 tests in 1.000s

But with --parallel=subprocess (and BZR_CONCURRENCY=2):

    blackbox.test_log.TestLogTags.test_log_with_tags                                                         OK        0ms
    blackbox.test_log.TestLogTags.test_merged_log_with_tags                                                  OK        0ms
    ----------------------------------------------------------------------
    Ran 2 tests in 3.531s

The timer is doing *something* as occasionally it thinks a test takes 15ms rather than 0ms, but clearly this is not actually timing how long the test is taking.

Does --parallel=fork have a similar issue? It appears to be using the same subunit.test_results.AutoTimingTestResultDecorator which I presume is responsible.