nunit-dev team mailing list archive

["Charlie Poole" <charlie@xxxxxxxxx>]

Hi All,

The bug in TestCaseSourceProvider pointed out by Andy
also showed up a Gui problem.

If there are no data providers that say they have cases
for a parameterized test case, it shows up as an error.

However, if a provider claims to have test cases but
later doesn't provide any the test shows up as inconclusive
unless the provider itself forces it to an error state.
For example, TestCaseSourceProvider does this in several
cases - for example, if the named source is not found.

This looseness was added intentionally to support theories,
where the data may not meet assumptions. But in the 
implementation, it's leaking over to normal tests.

Here's what I propose to implement...

For Tests: 
If there are no test cases at all, the test should show
up as an error - not inconclusive. If the provider gives
an error message, we'll use that. Otherwise we'll use
a generic message. I think the inconclusive result is
actually a bug in this case.

For Theories:
If no test cases were passed to the method, I think the 
theory itself should show as an error just as for a test.

If test cases were passed and they all failed the
assumptions of the theory, then the theory itself
should show as a failure. This is a change, but it's
what we talked about doing earlier anyway. The individual
cases under the theory would still show as inconclusive.

These changes could "break" some tests, but they are
probably already broken - just not reporting it!

What do you think?

Charlie