How do I run unit tests with dotnet test
if I have multiple test libraries in a code base?
I can run dotnet test
, and it will find and run all tests even across multiple libraries, but it runs and reports each test library run independently:
$ dotnet test
Test run for C:\Users\mark\Documents\Redacted.Test\bin\Debug\netcoreapp2.1\Redacted.Test.dll(.NETCoreApp,Version=v2.1)
Test run for C:\Users\mark\Documents\Redacted\Redacted.SqlAccess.Test\bin\Debug\netcoreapp2.1\Redacted.SqlAccess.Test.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Starting test execution, please wait...
Test Run Successful.
Total tests: 59
Passed: 59
Total time: 3.1779 Seconds
Test run for C:\Users\mark\Documents\Redacted\Redacted.RestApi.Tests\bin\Debug\netcoreapp2.1\Redacted.RestApi.Tests.dll(.NETCoreApp,Version=v2.1)
Microsoft (R) Test Execution Command Line Tool Version 16.2.0-preview-20190606-02
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Test Run Successful.
Total tests: 99
Passed: 99
Total time: 9.8155 Seconds
Test Run Successful.
Total tests: 25
Passed: 25
Total time: 21.2894 Seconds
In this example, there's two test libraries, so I get two test result outputs.
This may work OK if the code has already been compiled, but in a clean build, there's going to be a lot of output from the compiler. This can easily cause one of the test run summaries to scroll past the visible part of the screen.
That's a problem if that test run fails.
How can I collapse all the unit tests to a single pass/fail summary?
On .NET 4.x, I could, for instance, use xUnit.net's console runner to run all test libraries as a single suite:
$ ./packages/xunit.runner.console.2.4.0/tools/net461/xunit.console BookingApi.UnitTests/bin/Debug/Ploeh.Samples.Booking
Api.UnitTests.dll BookingApi.SqlTests/bin/Debug/Ploeh.Samples.BookingApi.SqlTests.dll
xUnit.net Console Runner v2.4.0 (64-bit Desktop .NET 4.6.1, runtime: 4.0.30319.42000)
Discovering: Ploeh.Samples.BookingApi.UnitTests
Discovered: Ploeh.Samples.BookingApi.UnitTests
Starting: Ploeh.Samples.BookingApi.UnitTests
Finished: Ploeh.Samples.BookingApi.UnitTests
Discovering: Ploeh.Samples.BookingApi.SqlTests
Discovered: Ploeh.Samples.BookingApi.SqlTests
Starting: Ploeh.Samples.BookingApi.SqlTests
Finished: Ploeh.Samples.BookingApi.SqlTests
=== TEST EXECUTION SUMMARY ===
Ploeh.Samples.BookingApi.SqlTests Total: 3, Errors: 0, Failed: 0, Skipped: 0, Time: 3.816s
Ploeh.Samples.BookingApi.UnitTests Total: 7, Errors: 0, Failed: 0, Skipped: 0, Time: 0.295s
-- - - - ------
GRAND TOTAL: 10 0 0 0 4.111s (5.565s)
Notice how this produces a single summary at the bottom of the screen, so that I can immediately see if my tests passed or failed.
Use dotnet vstest
to run multiple assemblies.
PS> dotnet vstest --help
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation. All rights reserved.
Usage: vstest.console.exe [Arguments] [Options] [[--] <RunSettings arguments>...]]
Description: Runs tests from the specified files.
Arguments:
[TestFileNames]
Run tests from the specified files. Separate multiple test file names
by spaces.
Examples: mytestproject.dll
mytestproject.dll myothertestproject.exe
...
Note that this method requires that you point at compiled assemblies (as opposed to dotnet test
, which wants you to point at project files, and will optionally build things first for you).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With