nunit-core team mailing list archive

[Dave Kipfer <david.kipfer@xxxxxxxxxxxx>]

Hi Charlie, Simone,

I find it misleading as the "test-step" XML element, by all appearances,
maps to the "Test" attribute in the class, yet the time elapsed includes
"SetUp" and "TearDown".  Similarly, "Maxtime" is an attribute against
the "Test", not "SetUp" and "TearDown", yet they can affect those
results as well.  Take a look at the documentation for Maxtime.  The
language used indicates the timing is against the "Test" method and
there are no notes stating otherwise.

I find it problematic as the only way I can see to hide or forgive the
performance of "SetUp" and "TearDown" would be to structure the test
such that those activities are placed in additional test steps in order
to get a timing for the actual stuff I want to time (i.e. what's in
"Test").  This results in superfluous test steps in the report which
really aren't important in the overall scheme of the test procedure.

If there is some way of suppressing the reporting of test steps, that
may alleviate that problem. Otherwise, the time should be a little more
specific:

1) In addition to "time", there could be "test-time", "setup-time", and "teardown-time" attributes in the XML report.
2) Attributes such as "Maxtime" could be further parameterized to give direction as to which part's timing is critical, since it applies to more than the test step.


Thanks,
Dave


-----Original Message-----
From: bounces@xxxxxxxxxxxxx [mailto:bounces@xxxxxxxxxxxxx] On Behalf Of Charlie Poole
Sent: Friday, October 26, 2012 2:20 PM
To: Kipfer, David
Subject: Re: [Nunit-core] [Bug 1071819] Re: Test time includes execution timeof Setup and Teardown methods

Hi Dave,

This has been the historical behavior of NUnit for a long time. SetUp
and TearDown are considered an integral part of the test and we have
generally felt that refactoring by moving some setup from SetUp to the
test itself - or vice versa - should not change the time reported.

As Simone suggests, can you give us some example of why this presents a
problem for you?

Charlie

On Fri, Oct 26, 2012 at 10:44 AM, Simone Busoli <1071819@xxxxxxxxxxxxxxxxxx> wrote:
> Hi Dave,
>
> it would be useful to know the reasons why you think that this 
> behavior is misleading and problematic.
>
> --
> You received this bug notification because you are a member of NUnit 
> Developers, which is subscribed to NUnit V2.
> https://bugs.launchpad.net/bugs/1071819
>
> Title:
>   Test time includes execution time of Setup and Teardown methods
>
> Status in NUnit V2 Test Framework:
>   New
>
> Bug description:
>   The calculated time of a test includes its SetUp and TearDown methods.
>
>   This includes:
>
>   The "time" attribute in the "test-case" element in an XML report is
>   not limited to the execution time of the decorated test method.
>
>   The "MaxTime" attribute fails on this same value (time of Setup + time
>   of teardown + time of test).  This is first of all misleading, and
>   secondly problematic.
>
>   A work around would appear to be to avoid Setup and Teardown methods
>   (as has been suggested for other reasons by some) and do
>   setup/teardown in the scope of the fixture or the test class
>   constructor/destructor.  In many cases, inlining setup and teardown
>   within the test case itself is not a feasible workaround.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1071819/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~nunit-core
> Post to     : nunit-core@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nunit-core
> More help   : https://help.launchpad.net/ListHelp

--
You received this bug notification because you are subscribed to the bug report.
https://bugs.launchpad.net/bugs/1071819

Title:
  Test time includes execution time of Setup and Teardown methods

Status in NUnit V2 Test Framework:
  New

Bug description:
  The calculated time of a test includes its SetUp and TearDown methods.

  This includes:

  The "time" attribute in the "test-case" element in an XML report is
  not limited to the execution time of the decorated test method.

  The "MaxTime" attribute fails on this same value (time of Setup + time
  of teardown + time of test).  This is first of all misleading, and
  secondly problematic.

  A work around would appear to be to avoid Setup and Teardown methods
  (as has been suggested for other reasons by some) and do
  setup/teardown in the scope of the fixture or the test class
  constructor/destructor.  In many cases, inlining setup and teardown
  within the test case itself is not a feasible workaround.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nunitv2/+bug/1071819/+subscriptions


The information contained in this e-mail message is PRIVATE. It may contain confidential information and may be legally privileged. It is intended for the exclusive use of the addressee(s). If you are not the intended recipient, you are hereby notified that any dissemination, distribution or reproduction of this communication is strictly prohibited. If the intended recipient(s) cannot be reached or if a transmission problem has occurred, please notify the sender immediately by return e-mail and destroy all copies of this message. 
Thank you.

-- 
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.
https://bugs.launchpad.net/bugs/1071819

Title:
  Test time includes execution time of Setup and Teardown methods

Status in NUnit V2 Test Framework:
  New

Bug description:
  The calculated time of a test includes its SetUp and TearDown methods.

  This includes:

  The "time" attribute in the "test-case" element in an XML report is
  not limited to the execution time of the decorated test method.

  The "MaxTime" attribute fails on this same value (time of Setup + time
  of teardown + time of test).  This is first of all misleading, and
  secondly problematic.

  A work around would appear to be to avoid Setup and Teardown methods
  (as has been suggested for other reasons by some) and do
  setup/teardown in the scope of the fixture or the test class
  constructor/destructor.  In many cases, inlining setup and teardown
  within the test case itself is not a feasible workaround.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nunitv2/+bug/1071819/+subscriptions