Why am I getting different behaviour when clicking Resharper "Run all tests" button vs using the keyboard shortcut command?

Question

The code I'm unit testing refers to an appsetting in the app.config file. To cater for this, I've added an app.config file to my unit tests project. If I click the "Run All Tests" icon in the Unit Test Sessions window, all my tests pass.

I have mapped the "ReSharper.ReSharper_UnitTest_RunSolution" command to Ctrl+Shift+Alt+U. If I run the tests by pressing this combination, the tests all run, but they fail to find the appsetting, which comes through as null.

I'm assuming this means that the button click runs under the context of the test project, whilst the command does not, but I can't quite work out what the command is doing.

Have I mapped the wrong command?

EDIT 1: I've also tried using the keyboard shortcut Alt-RUN (Resharper > Unit Tests > Run All), as well as clicking the menus manually, and found that this also causes all unit tests to not find the appsetting and therefore fail. Clicking the Run All Tests icon in Unit Test Sessions (the double green arrow) continues to work fine.

EDIT 2: I realised I should probably be mocking a separate class that fetches appsettings from the config file anyway, so this is what I'm now doing. So now there is no dependence on the config file when unit testing.

Answer 1

There are two things going on here. Firstly, the Run All Tests icon in the sessions window runs all tests in the session, while the Run All Tests menu item runs all tests in the solution. Slightly confusing that they've got the same name, but it does make sense given the context. This is why they give different results.

Secondly, when running all tests in a solution, the app setting might not get found. This is due to an optimisation the test runner makes which runs all tests in the same AppDomain. This avoids the overhead of creating a new AppDomain for each assembly, but has the downside that only one app.config will be used for all assemblies. If it picks the wrong one, your app setting is lost.

You can disable this by unchecking ReSharper » Options » Unit Testing » "Use separate AppDomain for each assembly with tests". Ideally, it should be disabled if any project has an app.config - I've added a feature request you can vote for and track: https://youtrack.jetbrains.com/issue/RSRP-428958