How to pass secrets into the test execution context from VSTS

Question

Values that need to be passed into the test execution context of a test typically get passed via "variables" in VSTS. These typically show up as environment variables.

These values may need to be kept secret (client secret from a service principal, password, etc). However, if you click the "lock" button in the VSTS Variables window for a given variable, it will not get exported into the test execution context as an environment variable (unlike non-secret variables), by design.

How do you pass secret variables and retrieve them in a test?

Answer 1

Secret variables need to be passed in as parameters to the vstest.exe process as "test run parameters". These will be substituted into values from the .runsettings file during test execution.

The high-level steps are:

1. Add a .runsettings file to your Visual Studio project
2. Get the secret value in your C# code
3. Create a VSTS build variable for your secret
4. Set your VSTS test task to use the .runsettings file you created
5. Explicitly pass in the build variable using the same name as the .runsettings file's key

Add a .runsettings file

1. Create an XML file with the extension "runsettings" (*.runsettings) and add it to your project

2. Add the following XML to the file. Note that it doesn't matter what you set the "value" to. It is going to get overridden by the VSTS test task anyway. Set it to a dummy value or to something that will cause the test to succeed locally.

   <?xml version="1.0" encoding="utf-8"?>
   <RunSettings>
       <TestRunParameters>
           <Parameter name="CLIENT_SECRET" value="secret" />
       </TestRunParameters>
   </RunSettings>
   

Get the secret value from your test code

1. Add a "TestContext" public property to your test class. This value will get set at runtime by the test runner. Note that this property MUST BE named TestContext.

   public TestContext TestContext { get; set; }
   

2. Retrieve your secret from the TestContext variable

   string clientSecret = TestContext.Properties["CLIENT_SECRET"].ToString();
   

Create a VSTS build variable for your secret

1. Navigate to your build definition
2. Click "Edit build definition"
3. Click "Variables"
4. Click "Add"
5. Name your variable & set its value
6. Click the lock icon at the end of the "value" field to set the value to secret

Set your VSTS test task to use the .runsettings file you created

1. Navigate to your build definition
2. Click "Edit build definition"
3. Click the "Tasks" button
4. Select your "test" task (or add one if there is not one)

5. Set the "Settings file" to your .runsettings file

   $/project/unitTest.runsettings
   

Explicitly pass in the build variable using the same name as the .runsettings file's key

1. Navigate to your build definition
2. Click "Edit build definition"
3. Click the "Tasks" button
4. Select your "test" task

5. In the "Override test run parameters" dialog, add the following (matching the spelling/case of the key you put in the .runsettings file for the argument name & the spelling/case of the test variable you added to the build definition for the value):

   -CLIENT_SECRET $(CLIENT_SECRET)