nunit-core team mailing list archive

[Charlie Poole <charlie@xxxxxxxxx>]

On Tue, Oct 2, 2012 at 1:03 PM, Simone Busoli
<1057981@xxxxxxxxxxxxxxxxxx> wrote:
> To summarize, the async feature works under NUnit built for CLR 2, of
> course it will use an agent to run tests built for CLR > 2.
>
> There is a sort of infrastructure problem that I posed already which
> concerns how to build and test the async thing. Currently I didn't make any
> explicit choice and the build fails if you try to build for CLR < 4,
> because it tries to build the .NET 4.5 test assembly.
> So far therefore you can successfully run a build with net-4.0, which will
> run the new tests as well. Of course I also ran them under NUnit built for
> CLR 2 but it's a manual process: exclude the tests from the build, build
> for CLR 2, invoke nunit-console.exe (CLR 2) on the 4.5 assembly previously
> built.

Yes, that's the infrastructure change I was hoping to avoid by postponing to
NUnit 3.0. :-) But never mind... you convinced me.

> Ideally we would have to come up with a way to say that even when we build
> for < 4.5 (there's no 4.5 now but only 4.0 as NAnt doesn't support it yet),
> that assembly is still built for 4.5 so that tests can be run on it with
> NUnit built for .NET < 4.5. I hope it makes sense. I would like to clarify
> this with Charlie.

What we need is a .NET 4.5 config. This guy has one...
http://stackoverflow.com/questions/11837795/building-net-4-5-projects-with-nant

> Also one question for Charlie: in order to see if tests are run correctly I
> created some fixtures which are not supposed to be run directly but via my
> own test code. In order to accomplish that I marked the suite as ignored,
> but those tests then end up in the build results as ignored, which is not
> really the case 'cause I'm running them by some other means. I wanted to
> figure out if there was a way to make NUnit /ignore/ those tests so that
> they don't show up as ignored.

Put those tests in the test-assembly dll. That's what it's for. To see how
it's used by other tests, just pick any class there and find references.

Charlie

> Simone
>
>
> On Tue, Oct 2, 2012 at 9:34 PM, Charlie Poole <charlie@xxxxxxxxx> wrote:
>
>> OK then, so you're safe. I know that their are various express releases and
>> some of them are limited in what they support.
>>
>> I haven't loaded and tested that code yet. But the net45 test program is
>> just a set of tests, not NUnit. It is of course built for .NET 4.5. Simone
>> is working on making sure it will run under NUnit so it may not yet be
>> complete. Note that some code in nunit.core.tests is under the control
>> of #if statements, which keep it from being compiled when not wanted.
>>
>> Charlie
>>
>> On Tue, Oct 2, 2012 at 12:03 PM, Jv <jv.ravichandran@xxxxxxxxx> wrote:
>> > Yes, it does, attached is the project properties page but the Express
>> > distribution says it is meant for evaluation purposes only. I do not know
>> > what help it can be from your perspective.
>> >
>> > Yes, but the Reflection class (in nunit.core.tests.net45) uses
>> > LinqExpressions. Will this not pose a problem if the target framework is
>> > 2.0? Sorry if this is a stupid question.
>> >
>> > Jv
>> >
>> > On Tue, Oct 2, 2012 at 11:23 PM, Charlie Poole <charlie@xxxxxxxxx>
>> > wrote:
>> >
>> >> Hi Jv,
>> >>
>> >> On Tue, Oct 2, 2012 at 9:54 AM, Jv <jv.ravichandran@xxxxxxxxx> wrote:
>> >> > Hi Charlie,
>> >> >
>> >> > No problems. Branched Simone's three times for different
>> reasons/tests.
>> >> All
>> >> > builds succeed. Building with VS 2012 Express edition for Windows
>> Desktop
>> >> > since it will be required by you later, I guess.
>> >>
>> >> No, that won't be required and won't even work I'm afraid.
>> >>
>> >> First, NUnit V2 will never be built for .NET 4.5. That's what 3.0 is
>> >> for.
>> >>
>> >> Second, what we are trying to test here is whether NUnit V2, built for
>> >> .NET 2.0, can run tests built for .NET 4.5, which use async. Simone has
>> >> used reflection to do it and has not used any 4.5 features.
>> >>
>> >> Of course, to test this, we need Tests built for .NET 4.5 but NUnit has
>> >> to be built as normally. Will the Express edition allow targeting .NET
>> 2.0
>> >> when you build NUnit?
>> >>
>> >> Charlie
>> >>
>> >> > I am working on the await scenarios, in an independent project file,
>> but
>> >> > with the .core dlls built of Simone's branch.
>> >> >
>> >> > My contribution, here, I guess will be to provide you with help
>> examples
>> >> > for the wiki and if something comes up related to the tests, will
>> post it
>> >> > here!
>> >> >
>> >> > A question though. Is it necessary for the Test method to be marked
>> >> Async?
>> >> > Since await returns a Task, can there be a scenario for the NUnit test
>> >> > method to not be Async ?
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Jv
>> >> >
>> >> > On Tue, Oct 2, 2012 at 9:40 PM, Charlie Poole <charlie@xxxxxxxxx>
>> wrote:
>> >> >
>> >> >> Hi Jv,
>> >> >>
>> >> >> You have to pull and build Simone's branch from source in order to be
>> >> >> able to work on this at all. As I remember from some previous work
>> >> >> you did, you were unable to do that on your machine. Is that still
>> >> >> a problem for you?
>> >> >>
>> >> >> Charlie
>> >> >>
>> >> >> On Tue, Oct 2, 2012 at 7:56 AM, Jv <jv.ravichandran@xxxxxxxxx>
>> wrote:
>> >> >> > Hi Charlie,
>> >> >> >
>> >> >> > 1. Will start with the scenarios outlined by you plus some of my
>> own.
>> >> >> Maybe
>> >> >> > they could feature in the help section.
>> >> >> >
>> >> >> > 2. Waiting for the merge to happen or can NUnit GUI work with
>> Simone's
>> >> >> > commit?
>> >> >> >
>> >> >> > Because I also want to test the GCConcurrent (for previous .net
>> >> versions)
>> >> >> > and serverGc (for .net 4.5) element and this can be done only
>> through
>> >> the
>> >> >> > App config file. I guess this is more important due to it being the
>> >> new
>> >> >> > feature in .Net 4.5.
>> >> >> >
>> >> >> > 3. The unmanaged aspect to it, where the above configuration
>> settings
>> >> can
>> >> >> > be overridden by the requesting app.What happens in the test (this
>> is
>> >> >> not a
>> >> >> > question) when the config settings can be overridden by a call to
>> >> >> > IsServerGC call from the unmanaged app. Is this  consideration
>> >> important?
>> >> >> > And to complicate things further, some processor.architectures are
>> not
>> >> >> > supported.
>> >> >> >
>> >> >> > I am not sure if this should be documented as a known issue in
>> NUnit
>> >> or
>> >> >> > should NUnit provide a hint when the Async tests are run or
>> executed?
>> >> >> >
>> >> >> > 4. The SynchronizationContext and the AsynchronizationContext can
>> be
>> >> >> quite
>> >> >> > tricky when GcCocurrent and ServerGC elements can both be used.
>> Please
>> >> >> > comment on this with reference to #2 above.
>> >> >> >
>> >> >> > Jv
>> >> >> >
>> >> >> > On Mon, Oct 1, 2012 at 3:37 AM, Charlie Poole <charlie@xxxxxxxxx>
>> >> wrote:
>> >> >> >
>> >> >> >> Sounds good! One thing we need is a test assembly built in VS2012,
>> >> >> >> which contains all sorts of combinations of test features, for
>> >> example,
>> >> >> >> test cases that use an async method, methods with timeout plus
>> async,
>> >> >> >> methods using testcasesource, etc. The existing NUnit tests have
>> lots
>> >> >> >> of examples but of course none of them can use async.
>> >> >> >>
>> >> >> >> This is actually something that would help us implement async
>>  tests
>> >> >> >> in all three projects... NUnit V2, NUnit 3.0 and NUnitLite
>> >> >> >>
>> >> >> >> Charlie
>> >> >> >>
>> >> >> >> On Sun, Sep 30, 2012 at 2:39 PM, Jv <jv.ravichandran@xxxxxxxxx>
>> >> wrote:
>> >> >> >> > Sure, Charlie, but this particular problem does require more
>> than
>> >> one
>> >> >> >> > person I thought and hence suggested to help. I will try to
>> >> >> independently
>> >> >> >> > simulate some similar async scenarios and then seek help on how
>> it
>> >> >> can be
>> >> >> >> > merged with Simone's work.
>> >> >> >> >
>> >> >> >> > Jv
>> >> >> >> >
>> >> >> >> > On Mon, Oct 1, 2012 at 2:45 AM, Charlie Poole <
>> charlie@xxxxxxxxx>
>> >> >> wrote:
>> >> >> >> >
>> >> >> >> >> Hi Jv,
>> >> >> >> >>
>> >> >> >> >> You should coordinate with Simone on this, since he has the
>> >> >> >> >> bug assigned. OTOH, there are tons of things to do other
>> >> >> >> >> than this if you want something of your own!
>> >> >> >> >>
>> >> >> >> >> Charlie
>> >> >> >> >>
>> >> >> >> >> On Sun, Sep 30, 2012 at 4:02 AM, Jv <jv.ravichandran@xxxxxxxxx
>> >
>> >> >> wrote:
>> >> >> >> >> > Since I am fairly free, I can work on this issue
>> independently
>> >> as a
>> >> >> >> >> > spike. Please point me to the appropriate files in NUnit v2.
>> >> >> >> >> >
>> >> >> >> >> > I will be working on finding if the async threads' context
>> can
>> >> be
>> >> >> >> >> > independently established.
>> >> >> >> >> >
>> >> >> >> >> > This will help in maintaining that tests "await"-ing a async
>> >> >> result is
>> >> >> >> >> > testing the right context and will also help in establishing
>> the
>> >> >> >> "void"
>> >> >> >> >> > context.
>> >> >> >> >> >
>> >> >> >> >> > Jv
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > You received this bug notification because you are
>> subscribed to
>> >> >> NUnit
>> >> >> >> >> > Extended Testing Platform.
>> >> >> >> >> > https://bugs.launchpad.net/bugs/1057981
>> >> >> >> >> >
>> >> >> >> >> > Title:
>> >> >> >> >> >   C#5 async tests are not supported
>> >> >> >> >> >
>> >> >> >> >> > To manage notifications about this bug go to:
>> >> >> >> >> >
>> >> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> You received this bug notification because you are a member of
>> >> NUnit
>> >> >> >> >> Developers, which is subscribed to NUnit V2.
>> >> >> >> >> https://bugs.launchpad.net/bugs/1057981
>> >> >> >> >>
>> >> >> >> >> Title:
>> >> >> >> >>   C#5 async tests are not supported
>> >> >> >> >>
>> >> >> >> >> Status in NUnit Test Framework:
>> >> >> >> >>   Triaged
>> >> >> >> >> Status in NUnitLite Testing Framework:
>> >> >> >> >>   Triaged
>> >> >> >> >> Status in NUnit V2 Test Framework:
>> >> >> >> >>   In Progress
>> >> >> >> >>
>> >> >> >> >> Bug description:
>> >> >> >> >>   If a method is marked async, nunit does not recognise it as a
>> >> test,
>> >> >> >> >>   and some test runners may incorrectly  pass it while it is
>> still
>> >> >> >> >>   running. These methods actually return a Task not void, and
>> the
>> >> >> test
>> >> >> >> >>   can only be passed if the task completes without a failure.
>> >> >> >> >>
>> >> >> >> >>   As detailed here
>> >> >> >> http://anthonysteele.co.uk/async-and-await-with-nunit
>> >> >> >> >>
>> >> >> >> >>   MSTest supports async tests
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx
>> >> >> >> >>
>> >> >> >> >> To manage notifications about this bug go to:
>> >> >> >> >>
>> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Regards,
>> >> >> >> >
>> >> >> >> > Ravichandran Jv
>> >> >> >> > http://ravichandranjv.blogspot.com
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > You received this bug notification because you are subscribed to
>> >> NUnit
>> >> >> >> > Extended Testing Platform.
>> >> >> >> > https://bugs.launchpad.net/bugs/1057981
>> >> >> >> >
>> >> >> >> > Title:
>> >> >> >> >   C#5 async tests are not supported
>> >> >> >> >
>> >> >> >> > To manage notifications about this bug go to:
>> >> >> >> >
>> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >> >> >>
>> >> >> >> --
>> >> >> >> You received this bug notification because you are a member of
>> NUnit
>> >> >> >> Developers, which is subscribed to NUnit V2.
>> >> >> >> https://bugs.launchpad.net/bugs/1057981
>> >> >> >>
>> >> >> >> Title:
>> >> >> >>   C#5 async tests are not supported
>> >> >> >>
>> >> >> >> Status in NUnit Test Framework:
>> >> >> >>   Triaged
>> >> >> >> Status in NUnitLite Testing Framework:
>> >> >> >>   Triaged
>> >> >> >> Status in NUnit V2 Test Framework:
>> >> >> >>   In Progress
>> >> >> >>
>> >> >> >> Bug description:
>> >> >> >>   If a method is marked async, nunit does not recognise it as a
>> test,
>> >> >> >>   and some test runners may incorrectly  pass it while it is still
>> >> >> >>   running. These methods actually return a Task not void, and the
>> >> test
>> >> >> >>   can only be passed if the task completes without a failure.
>> >> >> >>
>> >> >> >>   As detailed here
>> >> >> http://anthonysteele.co.uk/async-and-await-with-nunit
>> >> >> >>
>> >> >> >>   MSTest supports async tests
>> >> >> >>
>> >> >> >>
>> >> >>
>> >>
>> http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx
>> >> >> >>
>> >> >> >> To manage notifications about this bug go to:
>> >> >> >> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Regards,
>> >> >> >
>> >> >> > Ravichandran Jv
>> >> >> > http://ravichandranjv.blogspot.com
>> >> >> >
>> >> >> > --
>> >> >> > You received this bug notification because you are a member of
>> NUnit
>> >> >> > Developers, which is subscribed to NUnit V2.
>> >> >> > https://bugs.launchpad.net/bugs/1057981
>> >> >> >
>> >> >> > Title:
>> >> >> >   C#5 async tests are not supported
>> >> >> >
>> >> >> > Status in NUnit Test Framework:
>> >> >> >   Triaged
>> >> >> > Status in NUnitLite Testing Framework:
>> >> >> >   Triaged
>> >> >> > Status in NUnit V2 Test Framework:
>> >> >> >   In Progress
>> >> >> >
>> >> >> > Bug description:
>> >> >> >   If a method is marked async, nunit does not recognise it as a
>> test,
>> >> >> >   and some test runners may incorrectly  pass it while it is still
>> >> >> >   running. These methods actually return a Task not void, and the
>> test
>> >> >> >   can only be passed if the task completes without a failure.
>> >> >> >
>> >> >> >   As detailed here
>> >> http://anthonysteele.co.uk/async-and-await-with-nunit
>> >> >> >
>> >> >> >   MSTest supports async tests
>> >> >> >
>> >> >>
>> >>
>> http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx
>> >> >> >
>> >> >> > To manage notifications about this bug go to:
>> >> >> > https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Mailing list: https://launchpad.net/~nunit-core
>> >> >> > Post to     : nunit-core@xxxxxxxxxxxxxxxxxxx
>> >> >> > Unsubscribe : https://launchpad.net/~nunit-core
>> >> >> > More help   : https://help.launchpad.net/ListHelp
>> >> >>
>> >> >> --
>> >> >> You received this bug notification because you are a member of NUnit
>> >> >> Developers, which is subscribed to NUnit V2.
>> >> >> https://bugs.launchpad.net/bugs/1057981
>> >> >>
>> >> >> Title:
>> >> >>   C#5 async tests are not supported
>> >> >>
>> >> >> Status in NUnit Test Framework:
>> >> >>   Triaged
>> >> >> Status in NUnitLite Testing Framework:
>> >> >>   Triaged
>> >> >> Status in NUnit V2 Test Framework:
>> >> >>   In Progress
>> >> >>
>> >> >> Bug description:
>> >> >>   If a method is marked async, nunit does not recognise it as a test,
>> >> >>   and some test runners may incorrectly  pass it while it is still
>> >> >>   running. These methods actually return a Task not void, and the
>> test
>> >> >>   can only be passed if the task completes without a failure.
>> >> >>
>> >> >>   As detailed here
>> >> http://anthonysteele.co.uk/async-and-await-with-nunit
>> >> >>
>> >> >>   MSTest supports async tests
>> >> >>
>> >> >>
>> >>
>> http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx
>> >> >>
>> >> >> To manage notifications about this bug go to:
>> >> >> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Regards,
>> >> >
>> >> > Ravichandran Jv
>> >> > http://ravichandranjv.blogspot.com
>> >> >
>> >> > --
>> >> > You received this bug notification because you are subscribed to NUnit
>> >> > Extended Testing Platform.
>> >> > https://bugs.launchpad.net/bugs/1057981
>> >> >
>> >> > Title:
>> >> >   C#5 async tests are not supported
>> >> >
>> >> > To manage notifications about this bug go to:
>> >> > https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >>
>> >> --
>> >> You received this bug notification because you are a member of NUnit
>> >> Developers, which is subscribed to NUnit V2.
>> >> https://bugs.launchpad.net/bugs/1057981
>> >>
>> >> Title:
>> >>   C#5 async tests are not supported
>> >>
>> >> Status in NUnit Test Framework:
>> >>   Triaged
>> >> Status in NUnitLite Testing Framework:
>> >>   Triaged
>> >> Status in NUnit V2 Test Framework:
>> >>   In Progress
>> >>
>> >> Bug description:
>> >>   If a method is marked async, nunit does not recognise it as a test,
>> >>   and some test runners may incorrectly  pass it while it is still
>> >>   running. These methods actually return a Task not void, and the test
>> >>   can only be passed if the task completes without a failure.
>> >>
>> >>   As detailed here
>> http://anthonysteele.co.uk/async-and-await-with-nunit
>> >>
>> >>   MSTest supports async tests
>> >>
>> >>
>> http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx
>> >>
>> >> To manage notifications about this bug go to:
>> >> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>> >>
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Ravichandran Jv
>> > http://ravichandranjv.blogspot.com
>> >
>> >
>> > ** Attachment added: "targetframeworks.png"
>> >
>> https://bugs.launchpad.net/bugs/1057981/+attachment/3366113/+files/targetframeworks.png
>> >
>> > --
>> > You received this bug notification because you are subscribed to NUnit
>> > Extended Testing Platform.
>> > https://bugs.launchpad.net/bugs/1057981
>> >
>> > Title:
>> >   C#5 async tests are not supported
>> >
>> > To manage notifications about this bug go to:
>> > https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>>
>> --
>> You received this bug notification because you are a bug assignee.
>> https://bugs.launchpad.net/bugs/1057981
>>
>> Title:
>>   C#5 async tests are not supported
>>
>> Status in NUnit Test Framework:
>>   Triaged
>> Status in NUnitLite Testing Framework:
>>   Triaged
>> Status in NUnit V2 Test Framework:
>>   In Progress
>>
>> Bug description:
>>   If a method is marked async, nunit does not recognise it as a test,
>>   and some test runners may incorrectly  pass it while it is still
>>   running. These methods actually return a Task not void, and the test
>>   can only be passed if the task completes without a failure.
>>
>>   As detailed here http://anthonysteele.co.uk/async-and-await-with-nunit
>>
>>   MSTest supports async tests
>>
>> http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions
>>
>
> --
> You received this bug notification because you are subscribed to NUnit
> Extended Testing Platform.
> https://bugs.launchpad.net/bugs/1057981
>
> Title:
>   C#5 async tests are not supported
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions

-- 
You received this bug notification because you are a member of NUnit
Developers, which is subscribed to NUnit V2.
https://bugs.launchpad.net/bugs/1057981

Title:
  C#5 async tests are not supported

Status in NUnit Test Framework:
  Triaged
Status in NUnitLite Testing Framework:
  Triaged
Status in NUnit V2 Test Framework:
  In Progress

Bug description:
  If a method is marked async, nunit does not recognise it as a test,
  and some test runners may incorrectly  pass it while it is still
  running. These methods actually return a Task not void, and the test
  can only be passed if the task completes without a failure.

  As detailed here http://anthonysteele.co.uk/async-and-await-with-nunit

  MSTest supports async tests
  http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/13/what-s-new-for-mstest-unit-tests-in-visual-studio-11-beta.aspx

To manage notifications about this bug go to:
https://bugs.launchpad.net/nunit-3.0/+bug/1057981/+subscriptions