nunit-core team mailing list archive

[Charlie Poole <charlie@xxxxxxxxx>]

Hi Peter,

Of course, you're right about having Explicit - I must have blinked
(twice).

Regarding the category selection part of the problem, the crux of the issue
is this...

> /include=cat1+stress should perform tests having both categories, which is
> F1.StressTest() (as F1 has cat1, F1.StressTest() has cat1 from its parent F1 and
> stress on its own).

As implemented, categories filtering doesn't work that way by design. The
and filter was written to avoid matching multiple categories unless they
actually appeared on the same test. The change you suggest wouldn't
be a correction, but a new implementation.

Of course, it might be a better implementation, but the key question will be
whether it introduces breaking behavior for existing tests - at least
for NUnit 2.x.
I'll think about this as I work on the bug.

Charlie

On Tue, Jul 13, 2010 at 2:02 AM, Peter Arius <548841@xxxxxxxxxxxxxxxxxx> wrote:
> Hi Charlie,
>
> concerning #4:
>
> But there is an [Explicit] on both StressTest() methods in my example 8-).
> And yes, I meant that "+" means "and". I hope can make my point more
> understandable:
>
> The use case is "I want to run [Explicit] stress tests, but to narrow down the
> selection".
>
> Currently, one can neither achieve this with /include=stress+cat1,
> nor with /include=stress /run=MyNamespace.F1.
>
> /include=cat1+stress should perform tests having both categories, which is
> F1.StressTest() (as F1 has cat1, F1.StressTest() has cat1 from its parent F1 and
> stress on its own).
>
> Without [Explicit] on StressTest() methods, it works as expected:
> F1.StressTest() is performed.
>
> When selecting /include=stress, it works as expected, too: F1.StressTest() and
> F2.StressTest() are performed (selection by category beats [Explicit]).
>
> The point is that F1.StressTest does not have cat1 directly attached, but just
> indirectly from its parent. With /include=cat1+stress, it appears that
> "F1.StressTest has cat1 *from parent* and has stress on its own" is not enough
> to run the [Explicit] method.
>
> If one attaches cat1 and stress directly to F1.StressTest(), it's OK,
> too.
>
> AFAICS, the reason lies in AndFilter, which produces following call
> tree:
>
> AndFilter.Pass(test) // test == F1.Stress() TestMethod
>    +> CategoryFilter.Pass(test) // "cat1"
>        +> TestFilter.Pass(test)
>            +> CategoryFilter.Match(test) // "cat1"
>            <- true
>            or ...
>        <- true
>    <- true
>    and
>    +> CategoryFilter.Pass(test) // "stress"
>        +> TestFilter.Pass(test)
>            +> CategoryFilter.Match(test) // "stress"
>            <- false
>            or
>            +> TestFilter.MatchParent(test)
>                +> test.RunState != RunState.Explicit
>                <- false
>            <- false
>            or
>            +> TestFilter.MatchDescendant(test)
>            <- false
>    <- false
> <- false
>
> So, the "and" operation comes on top of the "Match(test) || MatchParent(test)
> || MatchDescendant(test)".
>
> My first guess is that one could evaluate the AndFilter.Pass like:
>
> if (test.RunState == RunState.Explicit)
> {
>    + require at least one filter to Match exactly
>    + require other filters to Match||MatchParent||MatchDescendant
> }
> else
> {
>    + normal and operation
> }
>
> but to me it's not obvious how to do this properly, and what all the side
> effects would be.
>
> Of course, this is not the same case Kuno brought up, but I think it's
> related because both deal with:
>
> + What does "explicitly selected" mean exactly?
> + especially with complex filters (AndFilter, AndFilter+NotFilter)
> + especially when some categories are on the parent, some on the children.
>
> Best regards,
>    Peter
>
> --
> [Explicit] does not get overridden if there is another category exclude
> https://bugs.launchpad.net/bugs/548841
> You received this bug notification because you are a member of NUnit
> Developers, which is subscribed to NUnit V2.
>

-- 
[Explicit] does not get overridden if there is another category exclude
https://bugs.launchpad.net/bugs/548841
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.

Status in NUnit V2 Test Framework: Triaged

Bug description:
Steps to reproduce:
   - Test code below ([1])
   - nunit-console.exe <assemby.dll> /include=T /exclude=B"

Expected result:
   - TestA() is executed

Actual result:
   - No test is executed

Remarks
   - Everything works as expected, if you remove the [Expected] attribute


[1] Test case

    [TestFixture, Explicit, Category("T")]
    public class Test
    {
        [Test, Category("A")]
        public void TestA() { Console.WriteLine("-A-"); }
        [Test, Category("B")]
        public void TestB() { Console.WriteLine("-B-"); }
    }