How to use different Service Connection for every stage in Azure Pipelines?

Question

When using multistage pipelines from yaml in Azure Pipelines and every stage is deploying resources to a separate environment, I'd like to use a dedicated service connection for each stage. In my case every stage is making use of the same deployment jobs, i.e. yaml templates. So I'm using a lot of variables that have specific values dependent on the environment. This works fine, except for the service connection.

Ideally, the variable that contains the service connection name, is added to the stage level like this:

stages:
- stage: Build
    # (Several build-stage specific jobs here)

- stage: DeployToDEV
  dependsOn: Build
  condition: succeeded()
  variables:
    AzureServiceConnection: 'AzureSubscription_DEV' # This seems like a logical solution
  jobs:
    # This job would ideally reside in a yaml template
    - job: DisplayDiagnostics
      pool:
        vmImage: 'Ubuntu-16.04'
      steps:
        - checkout: none
        - task: AzurePowerShell@4
          inputs:
            azureSubscription: $(AzureServiceConnection)
            scriptType: inlineScript
            inline: |
              Get-AzContext
            azurePowerShellVersion: LatestVersion

- stage: DeployToTST
  dependsOn: Build
  condition: succeeded()
  variables:
    AzureServiceConnection: 'AzureSubscription_TST' # Same variable, different value
  jobs:
    # (Same contents as DeployToDEV stage)

When this code snippet is executed, it results in the error message:

There was a resource authorization issue: "The pipeline is not valid. Job DisplayDiagnostics: Step AzurePowerShell input ConnectedServiceNameARM references service connection $(AzureServiceConnection) which could not be found. The service connection does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.

So, it probably can't expand the variable AzureServiceConnection soon enough when the run is started. But if that's indeed the case, then what's the alternative solution to make use of separate service connections for every stage?

One option that works for sure is setting the service connection name directly to all tasks, but that would involve duplicating identical yaml tasks for every stage, which I obviously want to avoid.

Anyone has a clue on this? Thanks in advance!

Answer 1

Currently you can not pass a variable as a serviceConnection. Apparently the service connection name is picked up on push/commit and whatever that is there will be picked up.

E.g. if you have a $(variable) it will pick $(variable) instead of the value.

Workaround I have used so far is to use a template for the steps at each stage and pass a different parameter with the serviceConnection.

Refer: https://github.com/venura9/azure-devops-yaml/blob/master/azure-pipelines.yml for a sample implementation. you are more than welcome to pull request with updates.

- stage: DEV
  displayName: 'DEV(CD)'
  condition: and(succeeded('BLD'), eq(variables['Build.SourceBranch'], 'refs/heads/develop'))
  dependsOn: 
   - BLD
  variables: 
    stage: 'dev'
  jobs:

  - job: Primary_AustraliaSouthEast
    pool:
      vmImage: $(vmImage)
    steps:
    - template: 'pipelines/infrastructure/deploy.yml'
      parameters: {type: 'primary', spn: 'SuperServicePrincipal', location: 'australiasoutheast'}
    - template: 'pipelines/application/deploy.yml'
      parameters: {type: 'primary', spn: 'SuperServicePrincipal'}