Running Universal Windows unit tests from the command line

Question

How do you run Universal Windows (UWP) unit test projects from the command line?

I am using the MSTestFramework.

Answer 1

Short answer:

vstest.console.exe /Platform:x64 AppPackages\UnitTestProject1_1.0.0.0_x64_Debug_Test\UnitTestProject1_1.0.0.0_x64_Debug.appx

Long answer:

1. Create the project by selecting Universal / Unit Test App (Universal Windows) template:

   

2. Build it with command line using in the folder where the solution file is

   msbuild /p:Platform=x64;Configuration=Debug

3. Try running the vstest.console.exe command above, in the short answer. It will fail, giving the following error message:

   error 0x800B0109: The root certificate of the signature in the app package or bundle must be trusted..
   For more details look into Event Viewer under Applications and Services Logs -> Microsoft -> Windows -> AppXDeployment-Server -> Microsoft-Windows-AppXDeploymentServer/Operational.
   

4. To be able to run tests from command line, you need to use a certificate which has trusted root, or make the certificate that generated by visual studio trusted. For the latter, double click UnitTestProject1_TemporaryKey.pfx file from windows explorer, and follow the import wizard default steps, but change two things:

   • Set Store Location to local machine:
   • Place the certificate to Trusted Root Certification Authorities store:
   • Finishing the wizard should say "The import was successful."
5. Try running vstest.console.exe using the parameters in the short answer, and now it should run all your tests.

Answer 2

I followed Ivan Marinov's answer, but I needed a purely command-line solution. Once you have your UWP Unit Test Project working and you are ready to automate, follow these steps :

(I named my solution Win10Universal and my unit test project Win10-UnitTests. You will need to substitute the names in my examples with your own)

1. Open Command Prompt as an administrator and navigate into the same directory as your solution.

2. Run MSBuild.exe on your solution.

>"C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe" Win10Universal.sln /p:Platform=x86;Configuration=Release

3. The build process will generate a directory similar to {SolutionDirectory}/AppPackages/{UnitTestTargetProjectName}/{Something_Test}. Navigate into this directory and inside you will see a .cer Security Certificate.

>cd AppPackages/Win10-UnitTests/Win10-UnitTests_1.1.0.0._x86_Test

4. Run CertMgr.exe on this generated certificate. This step will fail if you are not running Command Prompt with administrator privileges.

>"C:\Program Files (x86)\Windows Kits\10\bin\x86\certmgr.exe" -add Win10-UnitTests_1.1.0.0_x86_Release.cer -s -r localmachine root

5. Run VSTest.Console.exe on the .appx file in this directory.

>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" Win10-UnitTests_1.1.0.0_x86_Release.appx /Platform:x86

You should see your unit tests listed out in the window if you've done everything right! Hope this helps!

Answer 3

If both Ivan Marinov's and Kylaa's answers didn't work for you (like unfortunately happened to me), try using the appxrecipe of the UWP test app as the target file, and not the appx itself.

For example:

vstest.console.exe testProject.build.appxrecipe /platform:x64

https://github.com/Microsoft/vstest/issues/1393

https://github.com/Microsoft/vstest/issues/1477