nunit-core team mailing list archive

[Peter Arius <548841@xxxxxxxxxxxxxxxxxx>]

Hi Charlie,

concerning #4:

But there is an [Explicit] on both StressTest() methods in my example 8-).
And yes, I meant that "+" means "and". I hope can make my point more
understandable:

The use case is "I want to run [Explicit] stress tests, but to narrow down the 
selection".

Currently, one can neither achieve this with /include=stress+cat1, 
nor with /include=stress /run=MyNamespace.F1.

/include=cat1+stress should perform tests having both categories, which is 
F1.StressTest() (as F1 has cat1, F1.StressTest() has cat1 from its parent F1 and
stress on its own).

Without [Explicit] on StressTest() methods, it works as expected: 
F1.StressTest() is performed.

When selecting /include=stress, it works as expected, too: F1.StressTest() and 
F2.StressTest() are performed (selection by category beats [Explicit]).

The point is that F1.StressTest does not have cat1 directly attached, but just 
indirectly from its parent. With /include=cat1+stress, it appears that 
"F1.StressTest has cat1 *from parent* and has stress on its own" is not enough 
to run the [Explicit] method.

If one attaches cat1 and stress directly to F1.StressTest(), it's OK,
too.

AFAICS, the reason lies in AndFilter, which produces following call
tree:

AndFilter.Pass(test) // test == F1.Stress() TestMethod
    +> CategoryFilter.Pass(test) // "cat1"
        +> TestFilter.Pass(test)
            +> CategoryFilter.Match(test) // "cat1"
            <- true
            or ...
        <- true
    <- true
    and
    +> CategoryFilter.Pass(test) // "stress"
        +> TestFilter.Pass(test)
            +> CategoryFilter.Match(test) // "stress"
            <- false
            or
            +> TestFilter.MatchParent(test)
                +> test.RunState != RunState.Explicit
                <- false
            <- false
            or
            +> TestFilter.MatchDescendant(test)
            <- false
    <- false
<- false

So, the "and" operation comes on top of the "Match(test) || MatchParent(test) 
|| MatchDescendant(test)".

My first guess is that one could evaluate the AndFilter.Pass like:

if (test.RunState == RunState.Explicit)
{
    + require at least one filter to Match exactly
    + require other filters to Match||MatchParent||MatchDescendant
}
else
{
    + normal and operation
}
    
but to me it's not obvious how to do this properly, and what all the side
effects would be.

Of course, this is not the same case Kuno brought up, but I think it's
related because both deal with:

+ What does "explicitly selected" mean exactly?
+ especially with complex filters (AndFilter, AndFilter+NotFilter)
+ especially when some categories are on the parent, some on the children.

Best regards,
    Peter

-- 
[Explicit] does not get overridden if there is another category exclude
https://bugs.launchpad.net/bugs/548841
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.

Status in NUnit V2 Test Framework: Triaged

Bug description:
Steps to reproduce:
   - Test code below ([1])
   - nunit-console.exe <assemby.dll> /include=T /exclude=B"

Expected result:
   - TestA() is executed

Actual result:
   - No test is executed

Remarks
   - Everything works as expected, if you remove the [Expected] attribute


[1] Test case

    [TestFixture, Explicit, Category("T")]
    public class Test
    {
        [Test, Category("A")]
        public void TestA() { Console.WriteLine("-A-"); }
        [Test, Category("B")]
        public void TestB() { Console.WriteLine("-B-"); }
    }