nunit-core team mailing list archive

[Charlie Poole <question238839@xxxxxxxxxxxxxxxxxxxxx>]

Question #238839 on NUnit Framework changed:
https://answers.launchpad.net/nunit-3.0/+question/238839

    Status: Open => Answered

Charlie Poole proposed the following answer:
Now that we are moving away from Launchpad, it's better to ask
questions in our nunit-discuss google group. I've modified the project
page on Launchpad to indicate that.

Historically, NUnit only had the attributes executed and success.
Tests with executed=true were successful or failed depending on the
value of the success attribute. Those with executed=true didn't have a
success attrbute at all.

The result attribute was added at a later time, in order to cope with
new results beyond not run, success and failure. This gives the most
complete information on the result of a test and should be used by new
programs. The executed and success attribtes are kept for backward
compatibility.

Additionally, many report programs don't look at either of these. For
example, to list failures they simply find all tests with a <failure>
element.

To answer your conceptual questions:
  * No, NUnit does not consider ignored or inconclusive tests as successful.
  * A test suite is not considered successful unless it contains no
errors, failures or ignored items. Inconclusive tests don't affect the
result of a normal suite at all but may affect the result of a suite
representing a Theory - that is, if _all_ tests are inconclusive, the
theory is considered to fail. Essentially, each type of suite gets to
decide how it aggregates the results of its child tests.

NUnit 3.0 is using an entirely new result file format, by the way.

Charlie

On Thu, Nov 7, 2013 at 7:01 AM, Peter Brightman
<question238839@xxxxxxxxxxxxxxxxxxxxx> wrote:
> New question #238839 on NUnit Framework:
> https://answers.launchpad.net/nunit-3.0/+question/238839
>
> Hello all,
>
> i recently tried to count the results of a NUnit resultfile using xpath expressions. I have used version 2.5.10. What confuses me are the attributes of the test-case nodes. I have noticed the following attributes:
>
> executed
> result
> success
> time
> asserts
>
> So i have created some variables that count the results that way:
>
> <xsl:variable name="TestsAllCount" select="count(/test-results//test-case)"/>
> <xsl:variable name="FixturesFailed" select="count(/test-results//test-suite[failure][@type='TestFixture' and @result='Failure'])"/>
> <xsl:variable name="TestsFailed" select="count(/test-results//test-case[failure][@executed='True' and @success='False'])"/>
> <xsl:variable name="TestsPassed" select="count(/test-results//test-case[@executed='True' and @result='Success'])"/>
> <xsl:variable name="FixturesIgnored" select="count(/test-results//test-suite[@type='TestFixture' and @result='Ignored'])"/>
> <xsl:variable name="TestsIgnored" select="count(/test-results//test-case[@executed='False' and @result='Ignored'])"/>
> <xsl:variable name="FixturesInconclusive" select="count(/test-results//test-case[not(failure)][@executed='True'][@result='Inconclusive'])"/>
> <xsl:variable name="TestsInconclusive" select="count(/test-results//test-case[not(failure)][@executed='True'][@result='Inconclusive'])"/>
> <xsl:variable name="TestsNotExecuted" select="count(//test-case[@executed='False'])"/>
>
> Is this correct or not?
>
> I dig that ignored and inconclusive test have the executed-attribute set to 'False' but what for is the attribute "success"? Isn't a test successful even if it is ignored or inconclusive? As long as the attribute "success" just takes True|False as values, i wonder why i just can't use the attribute "result" that seems not to have just True|False as value but friendly names like "success", "failure" or "error" or "inconclusive" or "ignored". Is there any documentation about what values are set in what cases? What if a test-suite has 4 test-cases, one test is "Successful", another "Error", another "Ignored" and the last one is "Inconclusive". What value holds the attribute "result" of the appropriate test-suite of the mentioned test-cases? Does it contain all of the values of the test-cases in a comma-separated list? Please help. I am confused. Thank you.
>
> Peter
>
>
>
>
>
>
>
> --
> You received this question notification because you are a member of
> NUnit Core Developers, which is an answer contact for NUnit Framework.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~nunit-core
> Post to     : nunit-core@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~nunit-core
> More help   : https://help.launchpad.net/ListHelp

-- 
You received this question notification because you are a member of
NUnit Core Developers, which is an answer contact for NUnit Framework.