Azure DevOps file transform with vault secrets

Question

I need to deploy linux azure app service via azure devops. My configuration is stored in appsettings file and I need to substitute configuration values to the values stored in azure vault.

So I created variable group in artifacts, linked it to variables in pipeline and used FileTransform@2 to substitute appsettings values.

But it substitutes to null values. If I explicitly define variable value in pipeline by assigning some string value - it works fine.

Also I cannot use AzureRmWebAppDeployment@4 with JSONFiles, it does not work for linux deployment

What is the way of solving this?

here is pipeline code:

trigger:
  branches:
    include:
    - master
    - develop
    - release/*
  paths:
    include:
    - src/ConsumerBackEnd/*
    - src/ConsumerShared/*

variables:
  - name: poolName
    value: 'Private-Windows10'
  - name: azureRegisteredApp
    value: 'portal-devops'
  - name: workingDirectory
    value: '$(System.DefaultWorkingDirectory)/src/ConsumerBackEnd'
  - name: solutionDirectory
    value: '$(System.DefaultWorkingDirectory)/src'

stages:
- stage: Build
  displayName: Build stage

  jobs:
  - job: Build
    displayName: Build
    pool:
      name: $(poolName)
    
    variables:
    - group: ConsumerDevVariableGroup
    - name: 'Graph.GraphAppTenantId'
      value: '**************' #works fine
    - name: 'Graph.GraphAppClientId'
      value: '$[variables.GraphAppClientId]' #should take value from vault but injects null
    
    - task: FileTransform@2
      inputs:
        folderPath: '$(workingDirectory)'
        xmlTransformationRules: 
        jsonTargetFiles: '**/appsettings.json'

    - task: DotNetCoreCLI@2
      displayName: Nuget Restore
      inputs:
        command: 'restore'
        projects: '$(workingDirectory)/*.csproj'
        feedsToUse: 'config'
        nugetConfigPath: '$(solutionDirectory)/NuGet.config'
    
    - task: DotNetCoreCLI@2
      displayName: Build
      inputs:
        command: 'build'
        projects: |
          $(workingDirectory)/ConsumerBackEnd.csproj
        arguments: --output $(System.DefaultWorkingDirectory)/output

    - task: DotNetCoreCLI@2
      displayName: Publish
      inputs:
        command: 'publish'
        publishWebProjects: false
        projects: '$(workingDirectory)/ConsumerBackEnd.csproj'
        arguments: '-c Release -r linux-x64 --self-contained true --output $(System.DefaultWorkingDirectory)/publish_output'
        

#requires approval on pipeline
- stage: DeployDev
  displayName: DeployDev
  dependsOn: Build
  condition: succeeded()
  
  jobs:
    - deployment: DeployConsumerBackendAPIDev
      displayName: DeployConsumerBackendAPIDev
      environment: ConsumerBackendAPIDev
      pool:
        name: $(poolName)

      strategy:
        runOnce:
          deploy:
            steps:
            - task: AzureRmWebAppDeployment@4
              inputs:
                ConnectionType: 'AzureRM'
                azureSubscription: '$(azureRegisteredApp)'
                appType: 'webAppLinux'
                WebAppName: 'my-backend-dev'
                packageForLinux: '$(System.DefaultWorkingDirectory)/publish_output/**/*.zip'
                RuntimeStack: 'DOTNETCORE|LTS --configuration Release'

Answer 1

it appears that referencing group variable using runtime expression does not work with file transform task but macro syntax works fine

Microsoft documentation does not describe it well

so here is how it should be defined:

variables:
  #secrets
  - group: ConsumerDevVariableGroup
  - name: Graph.GraphAppTenantId
    value: $(GraphAppTenantId) #works fine
  - name: 'Graph.GraphAppClientId'
    value: '$[variables.GraphAppClientId]' #does not work

Answer 2

Azure DevOps file transform with vault secrets

To achieve this, you could use the task Replace Tokens to update the key's values with Azure vault secrets.

As test, I created a vault secret LeoVar1 in the Azure portal with test value 123456:

Then connect Azure key vault into azure devops pipeline.

Now, I use the Replace Tokens to replace the value of key webpages:Version in the web.config file with the vault secret in the Azure vault #{LeoVar1}#:

My test web.config file：

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <connectionStrings>
    <add name="DefaultConnection"
         connectionString="Data Source=(LocalDb)\\MSDB;DbFilename=aspcore-local.mdf;" />
  </connectionStrings>
  <appSettings>
    <add key="webpages:Version" value="#{LeoVar1}#" />
    <add key="webpages:Enabled" value="false" />
  </appSettings>
  <system.web>
    <authentication mode="None" />
    <compilation targetFramework="4.5" debug="true" />
  </system.web>
</configuration>

You will see the replace are finished successfully after this task ended：

You could check this document Store the app secrets in Azure Key Vault and use during Azure Pipelines for some details.

Update:

I just wandering why FileTransform@2 does not work with secrets

To make it work, you should make sure following things:

• The path of appsettings.json is correct. Makre sure it under the folder $(workingDirectory).
• The important is that make sure the name of the vault secret is the combination of the attribute name and its parent node.

For example, if we want change the DebugMode from enabled to disable in following json file:

{
  "Data": {
    "DefaultConnection": {
      "ConnectionString": "Data Source=(LocalDb)\\MSDB;AttachDbFilename=aspcore-local.mdf;"
    },
    "DebugMode": "enabled",
    "DBAccess": {
      "Administrators": ["Admin-1", "Admin-2"],
      "Users": ["Vendor-1", "vendor-3"]
    },
    "FeatureFlags": {
      "Preview": [
        {
          "newUI": "AllAccounts"
        },
        {
          "NewWelcomeMessage": "Newusers"
        }
      ]
    }
  }
}

We need defined the name of the vault secret to Data.DebugMode instead of DebugMode.

However, the point . is not supported by vault secret, it will give the error:

Please provide a valid secret name. Secret names can only contain alphanumeric characters and dashes.

If we test it with Variables in the pipeline, it works without any error:

Since the point . is not supported by vault secret, we could NOT replace the properties with parent node. That the reason why I suggest you use replace token task at first.