nunit-core team mailing list archive

[Charlie Poole <charlie@xxxxxxxxx>]

Hi Simon,

If this turns into a discussion, we should move it to the list. But lets
go another round here on a few points...

On Wed, Feb 2, 2011 at 2:00 AM, simon misys <711330@xxxxxxxxxxxxxxxxxx> wrote:
>> Such an assembly is not shown as a failure, but as "NotRunnable". [...]
>> It may also
>> be an issue that we use the same icon for it as for a test failure. [...]
>
> Hi Charlie, that was my point, that it is shown in the tree display with
> the failure icon (or, if you prefer, with the same icon as used for
> failures).  I did discover from its properties window that it is
> NotRunnable with no TestFixtures, but it was the use of the failure icon
> in the tree display itself that I was raising the issue for.
>
> (I also noted that the tree parent is also given the failure icon,
> though I don't know if you consider it unfortunate that its properties
> window does not give any indication that the cause is due to a
> NotRunnable child assembly.)

Yes, we migrate Non-Runnable up the tree because it's pretty severe
in some other cases - bad constructor for example. In fact, this may be
 a different thing, calling for a different category. I'll consider that.

> The user can discover/see that that the node is test-free, since it is
> not expandable, but I accept that it might be desirable to visually
> distinguish test-free assemblies from assemblies with tests that succeed
> in the tree display.  But, personally, I don't think it is desirable to
> make test-free assemblies visually equivalent to assemblies with tests
> that fail in the tree display.  One is "hmmm, we have no tests for this
> bit, but we should get some" and one is "argh!  we can't release this!!
> fire-fight!!!"  I guess choices might be:
>
> - use a new icon specifically for the purpose
> - use the existing question-mark icon
> - give the user the option to suppress the labelling of a test-free assembly with a non-success icon
>
>> I'm interested in knowing what
>> process you use to generate new project files when you add a new assembly.
>> Can't you simply filter out those with no tests at that point?
>
> Well, the easiest solution for us is to just manually add all of our
> existing Visual Studio projects' assemblies to the single new NUnit
> project.  That way, as team members add tests, we don't have to do
> anything to the NUnit project.  We don't expect to add many new Visual
> Studio projects, but we do expect to add many new tests, in a gradual
> process.  We want to avoid the situation whereby someone adds tests but
> forgets to make sure the corresponding assembly is part of the NUnit
> project's configurations (Debug/Release).  Does that answer your
> question?  Is that a reasonable use case?

So are you saying that you might create an assembly with no tests, but
later add tests to it? I guess that means you sometimes put tests into
the same assembly as the code they are testing.

I had been assuming that you would only create a new test assembly
if you had tests to put in it, so this is probably a use case that I wasn't
taking into account.

In any case, it's clear that something needs to be done here. I'll post
a comment when I have a better idea of what it is.

Charlie

> Thanks, Simon.
>
> --
> You received this bug notification because you are a member of NUnit
> Developers, which is subscribed to NUnit V2.
> https://bugs.launchpad.net/bugs/711330
>
> Title:
>  Test-free assembly given failure icon in tree display
>

-- 
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.
https://bugs.launchpad.net/bugs/711330

Title:
  Test-free assembly given failure icon in tree display

Status in NUnit V2 Test Framework:
  New

Bug description:
  A test-free assembly is shown with the failure icon, ie, an X with red
  background, in the GUI runner tree display, as is its parent(s).

  However, the progress bar is green, as expected (assuming all other
  tests pass).

  Why would we want to include a test-free assembly?  Because we want to
  create a single NUnit project for an existing large solution (40+
  Visual Studio projects) containing all of its assemblies and
  executables.  As we progressively add new tests to our product, we
  want NUnit to automatically pick them up without anyone having to
  remember to make sure the corresponding assembly/executable have been
  added to the NUnit project (and Debug/Release configurations).

  It is off-putting (to say the least) and misleading for NUnit label such test-free assemblies as failures in the tree display.
  It also contradicts what NUnit reports in the rest of the GUI (and when the NUnit project is run with the console runner).

  So, I think NUnit should not show test-free assemblies as failures in
  the tree display.  I think they should count as a success, ie, shown
  with a check with green background.

  Thanks.