How do I run unit tests with dotnet test if I have multiple test libraries in a code base?
I can run dotnet test, and it will find and run all tests even across multiple libraries, but it runs and reports each test library run independently:
$ dotnet test
Test run for C:\Users\mark\Documents\Redacted.Test\bin\Debug\netcoreapp2.1\Redacted.Test.dll(.NETCoreApp,Version=v2.1)
Test run for C:\Users\mark\Documents\Redacted\Redacted.SqlAccess.Test\bin\Debug\netcoreapp2.1\Redacted.SqlAccess.Test.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Starting test execution, please wait...
Test Run Successful.
Total tests: 59
Passed: 59
Total time: 3.1779 Seconds
Test run for C:\Users\mark\Documents\Redacted\Redacted.RestApi.Tests\bin\Debug\netcoreapp2.1\Redacted.RestApi.Tests.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Test Run Successful.
Total tests: 99
Passed: 99
Total time: 9.8155 Seconds
Test Run Successful.
Total tests: 25
Passed: 25
Total time: 21.2894 Seconds
In this example, there's two test libraries, so I get two test result outputs.
This may work OK if the code has already been compiled, but in a clean build, there's going to be a lot of output from the compiler. This can easily cause one of the test run summaries to scroll past the visible part of the screen.
That's a problem if that test run fails.
How can I collapse all the unit tests to a single pass/fail summary?
On .NET 4.x, I could, for instance, use xUnit.net's console runner to run all test libraries as a single suite:
$ ./packages/xunit.runner.console.2.4.0/tools/net461/xunit.console BookingApi.UnitTests/bin/Debug/Ploeh.Samples.Booking
Api.UnitTests.dll BookingApi.SqlTests/bin/Debug/Ploeh.Samples.BookingApi.SqlTests.dll
xUnit.net Console Runner v2.4.0 (64-bit Desktop .NET 4.6.1, runtime: 4.0.30319.42000)
Discovering: Ploeh.Samples.BookingApi.UnitTests
Discovered: Ploeh.Samples.BookingApi.UnitTests
Starting: Ploeh.Samples.BookingApi.UnitTests
Finished: Ploeh.Samples.BookingApi.UnitTests
Discovering: Ploeh.Samples.BookingApi.SqlTests
Discovered: Ploeh.Samples.BookingApi.SqlTests
Starting: Ploeh.Samples.BookingApi.SqlTests
Finished: Ploeh.Samples.BookingApi.SqlTests
=== TEST EXECUTION SUMMARY ===
Ploeh.Samples.BookingApi.SqlTests Total: 3, Errors: 0, Failed: 0, Skipped: 0, Time: 3.816s
Ploeh.Samples.BookingApi.UnitTests Total: 7, Errors: 0, Failed: 0, Skipped: 0, Time: 0.295s
-- - - - ------
GRAND TOTAL: 10 0 0 0 4.111s (5.565s)
Notice how this produces a single summary at the bottom of the screen, so that I can immediately see if my tests passed or failed.
Use dotnet vstest to run multiple assemblies.
PS> dotnet vstest --help
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation. All rights reserved.
Usage: vstest.console.exe [Arguments] [Options] [[--] <RunSettings arguments>...]]
Description: Runs tests from the specified files.
Arguments:
[TestFileNames]
Run tests from the specified files. Separate multiple test file names
by spaces.
Examples: mytestproject.dll
mytestproject.dll myothertestproject.exe
...
Note that this method requires that you point at compiled assemblies (as opposed to dotnet test, which wants you to point at project files, and will optionally build things first for you).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With