nunit-core team mailing list archive

[Charlie Poole <charlie@xxxxxxxxx>]

We'll definitely look at this bug more closely, just to understand what's
going on. For the moment, I'm just going to make some comments based on
past experience, for what it's worth...

1. Multiple construction of the attributes is, of course, not done directly
by NUnit. It comes from reflection calls.

2. Attribute construction should not normally be very costly.

3. In the past, we used caching of reflection results in order to avoid
this. We found that the cache itself cost a bigger performance hit than
just letting reflection do it's own thing. Any change we make for
performance purposes has to be measured by examining actual performance,
not just counting calls to the constructor.

4. Having non-NUnit attributes on a test method or class is somewhat
unusual. That is, most usage involves tests with NUnit attributes and
non-test classes and methods without any attributes. Of course, if you run
NUnit with a command line that causes it to examine non-test assemblies,
then you would see some different results.

5. NUnit 3.0 already does this somewhat differently (in fact so does NUnit
2.6.x). We will have to re-test this using the version of NUnit in which we
want to make any fix.

Charlie


On Thu, May 30, 2013 at 9:41 AM, Steve Heidstra <heidstra@xxxxxxxxx> wrote:

> Public bug reported:
>
> I have a simple code example that shows the bug attached below. In my
> solution I have overridden the TestCase attribute. The result of this was
> very slow performance. During the analysis of this perfomance issue it was
> found that the nUnit framework constructs attributes a lot more than
> expected:
> * an attribute applied to a class under test is constructed twice, even if
> the class is never constructed.
> * an attribute applied to a testfixture is constructed six times
> * an  attribute applied to a testmethod is constructed 13 times. This
> value varies as it exponentially increases when the number of tests
> increase.
>
> I use nUnit 2.5.10. I have run these tests with both the nUnit runner
> and the Resharper v6.1.1000.82 runner. They render the same result. But:
> if I change my tests to the MSTest framework and run it with the VS2010
> or VS2012 native runner, all attributes are constructed only once. If
> the MSTests are run with the Resharper runner another increase in the
> number of constructions can be seen.
>
> Below you will find the code that shows the issue:
>
> using System;
> using NUnit.Framework;
>
> namespace MyTests
> {
>     [AttributeUsage(AttributeTargets.All, AllowMultiple = true)]
>     public class WtfAttribute : Attribute
>     {
>         public static int CallCount = 0;
>
>         public WtfAttribute() //every time the attribute is constructed
> the count is incremented by 1.
>         {
>             CallCount++;
>         }
>     }
>
>     //this class is NEVER instantiated, but adding the attribute to it
> increases the callcount by 2. I expected it to be 1 due to reflection.
>     [Wtf]
>     public class MyDummyClass
>     {
>     }
>
>     [TestFixture]
>     //adding the attribute to MyTestClass increases the callcount by 6.
>     //[Wtf]
>     public class MyTestClass
>     {
>         [Test]
>         //adding the attribute to MyTest increases the callcount by 13.
>         //[Wtf]
>         public void MyTest()
>         {
>             Assert.AreEqual(1, WtfAttribute.CallCount, "CallCount = " +
> WtfAttribute.CallCount);
>         }
>     }
> }
>
> ** Affects: nunitv2
>      Importance: Undecided
>          Status: New
>
>
> ** Tags: attributes performance reflection
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1185704
>
> Title:
>   nUnit constructs TestCase attributes multiple times
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunitv2/+bug/1185704/+subscriptions
>

-- 
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.
https://bugs.launchpad.net/bugs/1185704

Title:
  nUnit constructs TestCase attributes multiple times

Status in NUnit V2 Test Framework:
  New

Bug description:
  I have a simple code example that shows the bug attached below. In my solution I have overridden the TestCase attribute. The result of this was very slow performance. During the analysis of this perfomance issue it was found that the nUnit framework constructs attributes a lot more than expected:
  * an attribute applied to a class under test is constructed twice, even if the class is never constructed. 
  * an attribute applied to a testfixture is constructed six times
  * an  attribute applied to a testmethod is constructed 13 times. This value varies as it exponentially increases when the number of tests increase.

  I use nUnit 2.5.10. I have run these tests with both the nUnit runner
  and the Resharper v6.1.1000.82 runner. They render the same result.
  But: if I change my tests to the MSTest framework and run it with the
  VS2010 or VS2012 native runner, all attributes are constructed only
  once. If the MSTests are run with the Resharper runner another
  increase in the number of constructions can be seen.

  Below you will find the code that shows the issue:

  using System;
  using NUnit.Framework;

  namespace MyTests
  {
      [AttributeUsage(AttributeTargets.All, AllowMultiple = true)]
      public class WtfAttribute : Attribute
      {
          public static int CallCount = 0;

          public WtfAttribute() //every time the attribute is constructed the count is incremented by 1.
          {
              CallCount++;
          }
      }

      //this class is NEVER instantiated, but adding the attribute to it increases the callcount by 2. I expected it to be 1 due to reflection.
      [Wtf]
      public class MyDummyClass
      {
      }

      [TestFixture]
      //adding the attribute to MyTestClass increases the callcount by 6.
      //[Wtf]
      public class MyTestClass
      {
          [Test]
          //adding the attribute to MyTest increases the callcount by 13.
          //[Wtf]
          public void MyTest()
          {
              Assert.AreEqual(1, WtfAttribute.CallCount, "CallCount = " + WtfAttribute.CallCount);
          }
      }
  }

To manage notifications about this bug go to:
https://bugs.launchpad.net/nunitv2/+bug/1185704/+subscriptions