nunit-core team mailing list archive

[Charlie Poole <charlie@xxxxxxxxx>]

Hi Sione,

On Wed, Mar 16, 2011 at 11:48 PM, Simone Busoli
<697329@xxxxxxxxxxxxxxxxxx> wrote:
> Fixing the bug is was I am trying to do :) But I'm not sure how it is
> supposed to work then, I would go with doing the less unintuitive thing.
> So, keeping the current implementation of using project configuration, when
> exactly should a project be excluded? I would say when:
>
>   - you specify /config:ConfigName
>   - there is a solution configuration called ConfigName
>   - there is a project with configuration called ConfigName
>   - the above project/ConfigName pair is disabled in that solution's
>   ConfigName configuration
>
> It's a heuristic that may solve the most common situations.

I agree that this is a reasonable implementation. And - as you point out -
it can only be achieved by looking at solution configurations, which we
currently ignore.

The root of the problem here is that the author of the bug is using the
solution directly on the command line. That's a reasonable thing to want
to do but - unfortunately - my own vision was that the solution would be
used on the command line for very simple cases. For more complex cases,
I imagined that one would save an nunit project from the solution and
edit it as needed. However, he demonstrated a fairly reasonable use case
for wanting this to work directly, so I think we should support it until/unless
the effort becomes too large for the benefit.

It seems that we need a graded series of test cases, from the most simple
up to the most complex we wish to handle. I'd say start with one that has
solution and project configs the same and no platform other than Any Cpu,
with one project disabled in a particular configuration. You could then
build back up toward the sort of example you created, up to the point
where it's not worth the effort. At that point, we can just document
the limitations.

Charlie

Charlie
> On Thu, Mar 17, 2011 at 02:51, Charlie Poole <charlie@xxxxxxxxx> wrote:
>
>> Hi Simone,
>>
>> Well, I told you this one was non-trivial. :-)
>>
>> I've been aware of the issue but it seemed like a significant effort
>> to make the change.
>> What's more, over the years that we have had this feature, nobody has
>> ever filed a
>> bug against it asking that we use solution configurations.
>>
>> OTOH, if you want to do a more correct implementation, I'm in favor of
>> it. However,
>> it might be good to first solve the immediate bug (disabled projects)
>> and develop some
>> infrastructure for tying solution configurations to project
>> configurations correctly.
>>
>> Since it's a potentially breaking change, I'd say that 3.0 is the
>> place to make it, if
>> we make it at all.
>>
>> One thing to keep in mind: it's always possible to save the solution
>> created as
>> an nunit project and make any changes desired. NUnit's own project
>> structure
>> does not have the notion of two levels of configuration at all.
>>
>> Charlie
>>
>>
>> On Wed, Mar 16, 2011 at 12:51 PM, Simone Busoli
>> <697329@xxxxxxxxxxxxxxxxxx> wrote:
>> > Looking at the code I noticed the only configurations we are taking into
>> > account are project configurations. This feature requires that we take
>> > solution configurations into account, which don't necessarily map to
>> > project configurations. The example is a bi contrived, so please bear
>> > with me:
>> >
>> > Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") =
>> "SolutionConfigurations",
>> "SolutionConfigurations\SolutionConfigurations.csproj",
>> "{C9F48484-42ED-4EAC-9434-8E15470C378A}"
>> > EndProject
>> > Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") =
>> "SolutionConfigurations2",
>> "SolutionConfigurations2\SolutionConfigurations2.csproj",
>> "{5FC3C757-FF4C-4956-B939-FDFB8255F698}"
>> > EndProject
>> > Global
>> >        GlobalSection(SolutionConfigurationPlatforms) = preSolution
>> >                SolutionDebug|SolutionAnyCPU =
>> SolutionDebug|SolutionAnyCPU
>> >                SolutionDebug|Solutionx86 = SolutionDebug|Solutionx86
>> >                SolutionRelease|SolutionAnyCPU =
>> SolutionRelease|SolutionAnyCPU
>> >                SolutionRelease|Solutionx86 = SolutionRelease|Solutionx86
>> >        EndGlobalSection
>> >        GlobalSection(ProjectConfigurationPlatforms) = postSolution
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionDebug|SolutionAnyCPU.ActiveCfg
>> = Debug|x86
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionDebug|Solutionx86.ActiveCfg
>> = Debug|x86
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionDebug|Solutionx86.Build.0 =
>> Debug|x86
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionRelease|SolutionAnyCPU.ActiveCfg
>> = Release|x86
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionRelease|SolutionAnyCPU.Build.0
>> = Release|x86
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionRelease|Solutionx86.ActiveCfg
>> = Release|x86
>> >
>>  {C9F48484-42ED-4EAC-9434-8E15470C378A}.SolutionRelease|Solutionx86.Build.0
>> = Release|x86
>> >
>>  {5FC3C757-FF4C-4956-B939-FDFB8255F698}.SolutionDebug|SolutionAnyCPU.ActiveCfg
>> = Debug|Any CPU
>> >
>>  {5FC3C757-FF4C-4956-B939-FDFB8255F698}.SolutionDebug|SolutionAnyCPU.Build.0
>> = Debug|Any CPU
>> >
>>  {5FC3C757-FF4C-4956-B939-FDFB8255F698}.SolutionDebug|Solutionx86.ActiveCfg
>> = Debug|Any CPU
>> >
>>  {5FC3C757-FF4C-4956-B939-FDFB8255F698}.SolutionRelease|SolutionAnyCPU.ActiveCfg
>> = Release|Any CPU
>> >
>>  {5FC3C757-FF4C-4956-B939-FDFB8255F698}.SolutionRelease|SolutionAnyCPU.Build.0
>> = Release|Any CPU
>> >
>>  {5FC3C757-FF4C-4956-B939-FDFB8255F698}.SolutionRelease|Solutionx86.ActiveCfg
>> = Release|Any CPU
>> >        EndGlobalSection
>> >        GlobalSection(SolutionProperties) = preSolution
>> >                HideSolutionNode = FALSE
>> >        EndGlobalSection
>> > EndGlobal
>> >
>> > As per the current implementation the NUnitProject resulting from the
>> > above solution file has 4 configurations: Debug|x86, Release|x86, Debug,
>> > Release (Any CPU is stripped from the project configuration name). It is
>> > slightly more than a coincidence if those configurations match those of
>> > the solution, and in fact in the above example they don't as I changed
>> > names of the solution configurations and platforms. For each solution
>> > configuration each project contained in the solution may have a
>> > different configuration, therefore relying on project configurations
>> > only is no longer enough.
>> >
>> > Therefore in the above example, for solution configuration:
>> >
>> > SolutionDebug|SolutionAnyCPU ->
>> >
>> >    Project SolutionConfigurations has configuration Debug|x86 but is not
>> built
>> >    Project SolutionConfigurations2 has configuration Debug|Any CPU
>> >
>> > SolutionDebug|Solutionx86 ->
>> >
>> >    Project SolutionConfigurations has configuration Debug|x86
>> >    Project SolutionConfigurations2 has configuration Debug|Any CPU but is
>> not built
>> >
>> > SolutionRelease|SolutionAnyCPU ->
>> >
>> >    Project SolutionConfigurations has configuration Release|x86
>> >    Project SolutionConfigurations2 has configuration Release|Any CPU
>> >
>> > SolutionRelease|Solutionx86 ->
>> >
>> >    Project SolutionConfigurations has configuration Release|x86
>> >    Project SolutionConfigurations2 has configuration Release|Any CPU but
>> is not built
>> >
>> > As usual the devil is in the details, so I would say the current
>> > implementation is wrong already as it assumes that project configs
>> > mirror solution configs.
>> >
>> > I think this would be a breaking change, as the /config switch until now
>> > was used (probably unconsciously) to specify project configurations, not
>> > solution configurations, and there is no name mapping between the two.
>> >
>> > Ideas?
>> >
>> > --
>> > You received this bug notification because you are a member of NUnit
>> > Developers, which is subscribed to NUnit V2.
>> > https://bugs.launchpad.net/bugs/697329
>> >
>> > Title:
>> >  NUnit tries to test disabled projects
>> >
>>
>> --
>> You received this bug notification because you are a bug assignee.
>> https://bugs.launchpad.net/bugs/697329
>>
>> Title:
>>  NUnit tries to test disabled projects
>>
>> Status in NUnit V2 Test Framework:
>>  Triaged
>>
>> Bug description:
>>  See discussion here: http://groups.google.com/group/nunit-
>>  discuss/browse_thread/thread/c1ab27746efb6d2c
>>
>>  I use the following command to run all unit tests in Visual Studio
>>  solutions:
>>
>>  nunit-console-x86 foo.sln /config=Release
>>
>>  That works great! Except:
>>
>>  Since some projects should not be built in the release configuration,
>>  they have been disabled in the release configuration (using Visual
>>  Studio's "Configuration Manager" - the "Build" checkbox is unchecked)

-- 
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.
https://bugs.launchpad.net/bugs/697329

Title:
  NUnit tries to test disabled projects

Status in NUnit V2 Test Framework:
  Triaged

Bug description:
  See discussion here: http://groups.google.com/group/nunit-
  discuss/browse_thread/thread/c1ab27746efb6d2c

  I use the following command to run all unit tests in Visual Studio
  solutions:

  nunit-console-x86 foo.sln /config=Release

  That works great! Except:

  Since some projects should not be built in the release configuration,
  they have been disabled in the release configuration (using Visual
  Studio's "Configuration Manager" - the "Build" checkbox is unchecked).

  The problem is that nunit still tries to test the disabled projects,
  which results in the following "File not found"-error:

  C:\buildsystem\bin\CruiseControl.NET-1.4.3.4023-patched\WorkingDirs\Foo-v2.8>nunit-console-x86 foo.sln /config=Release 
  NUnit version 2.5.9.10348 
  Copyright (C) 2002-2009 Charlie Poole. 
  Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov. 
  Copyright (C) 2000-2002 Philip Craig. 
  All Rights Reserved. 
  Runtime Environment - 
     OS Version: Microsoft Windows NT 6.1.7600.0 
    CLR Version: 2.0.50727.4927 ( Net 2.0 ) 
  ProcessModel: Default    DomainUsage: Default 
  Execution Runtime: Default 
  Could not find file 'C:\buildsystem\bin\CruiseControl.NET-1.4.3.4023-patched\WorkingDirs\Foo-v2.8\SomeDisabledProject\bin\Release\SomeDisabledProject.dll'.

  Since specifying the sln-file is very convenient, I'd prefer not
  having to specify single csproj-files or single dlls...

  Thanks, 
  Max