Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

NUnit 3: Forbid tests to run in parallel

Tags:

I have the latest NUnit(3.2.0) installed and I have all my tests run in parallel. It might look like desirable behavior but I didn't ask for it and actually it broke some of my tests. I have some initialization in [OneTimeSetUp] which is thread-dependent and it seems I can't do anything to force NUnit to run my tests sequentially. I've read the documentation and it states that by default tests aren't run in parallel but in fact they are!

Moreover, I've tried to add the following attribute: [assembly: Parallelizable(ParallelScope.None)] — no luck.

Does anybody know how to change this behavior?

P.S. I run it with ReSharper but also tried with MSVS add-in.


UPD: I'm using MVVM Light DispatcherHelper.Initialize()(inside[OneTimeSetUp]) to store the dispatcher object which is later used by a couple of tests. If threads are different(between a test and the setup method) then the action under test gets executed asynchronously and my tests fail.

I've checked the thread ids in different tests and they all are different.


UPD2: Excerpt from the documentation:

The NUnit 3.0 framework can run tests in parallel within an assembly. This is a completely separate facility from Engine Parallel Test Execution, although it is possible to use both in the same test run.

By default, no parallel execution takes place. Attributes are used to indicate which tests may run in parallel and how they relate to other tests.

If it doesn't mean the tests within an assembly should not be run in parallel until explicitly specified then what does it mean? And why [assembly: Parallelizable(ParallelScope.None)] has no effect on the tests parallel execution?


UPD3: Answer to the question might be found below but if you are stuck(as I was) with the DispatcherHelper.Initialize() you just need to remove this initialization from the OneTimeSetUp and put the following lines in every test that uses a dispatcher:

DispatcherHelper.Reset(); DispatcherHelper.Initialize(); 
like image 288
ixSci Avatar asked Mar 14 '16 08:03

ixSci


People also ask

Do NUnit tests run in parallel?

On BrowserStack, you can run multiple NUnit tests at the same time across various browser, device, and OS combinations. This is Parallel Testing.

Does MSTest run tests in parallel?

The biggest advantage of MSTest is that it allows parallelization at the method level. As opposed to the other frameworks which only allow parallelization at the class level. So, for example, If you have 100 test methods in 5 classes, MSTest will let you run 100 tests in parallel.

How do you do multiple runs of the same in NUnit?

To run the same test fixture twice, in parallel, you need to pass strings as parameters into the TestFixture constructor. For example, the following code runs the test fixture once against an iOS device on a certain version and with a certain device description and a second time against an Android device.

How does NUnit decide what order to run tests in?

There is no facility in NUnit to order tests globally. Tests with an OrderAttribute argument are started before any tests without the attribute. Ordered tests are started in ascending order of the order argument. Among tests with the same order value or without the attribute, execution order is indeterminate.


1 Answers

NUnit does not guarantee that all of your tests will run on the same thread, so the observation that your tests are running on different threads does not mean they are running in parallel.

The documentation only states that tests will run sequentially or in parallel. You may construe that this means they run on the same thread, but there are many reasons that the internal implementation might require tests to run on different threads. Timeout is an example, where we spawn a thread and kill it if the test times out, but there are many others.

Parallel test runs are new to NUnit 3, so the internal implementation changed from NUnit 2. An attribute that forces all tests within a thread to run on the same thread might be useful, so feel free to submit an enhancement request.

Sorry, I am unfamiliar with MVVM Light, so I can't suggest ways to marshal back to the OneTimeSetup thread.

Update - Since this is a common usage with web and async, the NUnit team has decided to provide an attribute that will demand tests be run on the same thread as the fixture's OneTimeSetup. This will be in the next release, either 3.4, or in a hotfix 3.2.1 release. If you want to track progress, see the issue and the pull request.

Update 2 - You can now add SingleThreadedAttribute to a TestFixture to indicate to the runner that the OneTimeSetUp, OneTimeTearDown and all the child tests must run on the same thread.

like image 86
Rob Prouse Avatar answered Sep 19 '22 15:09

Rob Prouse