nunit-core team mailing list archive

[Charlie Poole <charlie@xxxxxxxxx>]

** Changed in: nunitv2
       Status: Triaged => Fix Committed

-- 
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.
https://bugs.launchpad.net/bugs/899178

Title:
  Wrong failure message for parameterized tests that expect exceptions

Status in NUnit Test Framework:
  Triaged
Status in NUnit V2 Test Framework:
  Fix Committed

Bug description:
  This problem exists in both 2.5.10 and 2.6 Beta 2.  This problem
  happens regardless of how it is run, though we have been using the GUI
  runner.

  There are two classes, MyFixture and MySource.  MyFixture looks like
  this:

  [TestFixture]
  public class MyFixture
  {
  	[TestCaseSource(typeof(MySource), "Cases")]
  	public string BlahTest(string signifier)
  	{
  		switch (signifier)
  		{
  			case "A":
  				return "BLAH";

  			case "B":
  				throw new Exception("HERE");

  			case "C":
  				return "C";

  			default:
  				return null;
  		}
  	}
  }

  MySource looks like this:

  public class MySource
  {
  	public static IEnumerable<TestCaseData> Cases
  	{
  		get
  		{
  			yield return new TestCaseData("A").Returns("BLAH");
  			yield return new TestCaseData("B").Throws("System.Exception");
  			yield return new TestCaseData("C").Throws("System.Exception");
  			yield return new TestCaseData("D").Throws("System.Exception");
  		}
  	}
  }

  We expect that the first two tests will succeed and that the second
  two will fail, and that is the case.  However, the failure message on
  the third case is not the correct message.  We are supposed to see
  "System.Exception was expected".  However, we see 'Expected: null But
  was: "C"'.  The reason is that in the NUnit source, the ParameterSet
  class that retrieves test parameters from TestCaseData instances has
  the following on line 277:

  parms.Result = GetParm(source, PropertyNames.Result);

  This line is executed for every test case regardless of whether an
  exception is expected.  The setter for the Result property (lines
  146-7) has the following:

  result = value;
  hasExpectedResult = true;

  When this is consumed by the test runner, the runner thinks the test
  case has an expected result, when in fact it does not.  To fix this,
  the following line should be inserted immediately before line 277:

  if(parms.ExpectedExceptionName != null)

  This will make sure the Result is only set if an exception is not
  expected, thus preserving the behavior the developer intended.

  (Credit for discovering the solution to this problem goes to my son,
  David Churvis.)

To manage notifications about this bug go to:
https://bugs.launchpad.net/nunit-3.0/+bug/899178/+subscriptions