Unable to use output variables set in Azure DevOps pipeline

Question

I have a pipeline which uses a templated job as follows:

  - template: '/pipeline/templates/jobs/DeployInfraToRegion.yml@devops'
    parameters:
      ...

This job deploys a compiled Bicep template to Azure. This template creates a KeyVault and outputs its name as follows:

output sqlSecretsKeyVaultName string = keyvault.outputs.keyVaultName

The above template contains a single job named DeployInfrastructure:

jobs:
- deployment: DeployInfrastructure
  displayName: Deploy Infrastructure
  variables:
  ...

It runs a final task which takes all outputs and sets them as pipeline variables:

            - task: PowerShell@2
              name: SetArmOutputsForRegion
              displayName: "Set Arm outputs as variables"
              inputs:
                targetType: inline
                pwsh: true
                errorActionPreference: stop
                script: |
                  $armOutputString = '$(${{ parameters.armOutputVariable }})'

                  $armOutputObj = $armOutputString | convertfrom-json
                  $armOutputObj.PSObject.Properties | ForEach-Object {
                    $type = ($_.value.type).ToLower()
                    $key = $_.name
                    $value = $_.value.value

                    if ($type -eq "securestring") {
                        Write-Host "##vso[task.setvariable variable=$key;isOutput=true;issecret=true]$value"
                    } elseif ($type -eq "string") {
                        Write-Host "##vso[task.setvariable variable=$key;isOutput=true]$value"
                    } else {
                        Throw "Type '$type' not supported!"
                    }
                  }

Note that this task is named SetArmOutputsForRegion.

When this runs I can see in the pipeline out that this pipeline variable is being set as expected:

##[debug]Processed: ##vso[task.setvariable variable=sqlSecretsKeyVaultName;isOutput=true]expected-keyvault-name

After this templated job is run, the next job needs to use this output variable. I've configured it like this:

  - job: 'AddSecretsToKeyVault'
    dependsOn: DeployInfrastructure
    variables:
      keyVaultName: $[ dependencies.DeployInfrastructure.outputs['SetArmOutputsForRegion.sqlSecretsKeyVaultName'] ]
      ...

Note that this job depends on the job where the variables are set, and this is referenced as dependencies.DeployInfrastructure. The string is in the format [task name].[variable name] as per the documentation. However, when I try to access this variable it is always empty:

    - task: AzurePowerShell@4
      displayName: Ensure required secrets exist
      inputs:
        azureSubscription: MyConnection
        failOnStandardError: true
        azurePowerShellVersion: 'LatestVersion'
        scriptType: 'inlineScript'
        inline: |
          Write-Host "Vault name is $(keyVaultName)"

I feel at this point I've tried every possible variant of the variable declaration. I've triple checked all the names, I've had all my code sanity checked by others. I just cannot understand why these variables are not available to me. Posting here in the desperate hope somebody can explain what I'm doing wrong. Thanks in advance.

Update 1

I've now discovered documentation about deployment jobs, which apparently treat output variables differently. In my case I have a runOnce job, so I need to include the job name as a prefix. In other words, change...

    variables:
      keyVaultName: $[ dependencies.DeployInfrastructure.outputs['SetArmOutputsForRegion.sqlSecretsKeyVaultName'] ]

...to...

    variables:
      keyVaultName: $[ dependencies.DeployInfrastructure.outputs['DeployInfrastructure.SetArmOutputsForRegion.sqlSecretsKeyVaultName'] ]

However, having made this change, the variable value is still empty.

Could this have something to do with the variable being set by a templated job from another repo? I don't see how, but I honestly don't think I'm doing anything wrong :(

Answer 1

Finally worked this out. Eventually solved it with a neat trick I'd like to share.

In the job where you need to use the output variables, add the following variable:

variables:
      checkDependencies: $[ convertToJson(dependencies) ]

Then add the following task:

    - script: "echo '$(checkDependencies)'"
      name: checkDependencies

This will print the whole dependencies JSON object to your pipeline output, allowing you to inspect the outputs and avoid all guesswork. As described here, this object takes the format:

"dependencies": {
  "<STAGE_NAME>" : {
    "result": "Succeeded|SucceededWithIssues|Skipped|Failed|Canceled",
    "outputs": {
        "jobName.stepName.variableName": "value"
    }
  },
  "...": {
    // another stage
  }
}

In my particular case, it appears (by my reading) that the documentation I referenced in my Update 1 is wrong. Performing the above check told me that I need to use this format to retrieve my output variable value:

$[ dependencies.DeployInfrastructure.outputs['Deploy.SetArmOutputsForRegionEun.sqlSecretsKeyVaultName'] ]

This solved the problem in my case. The documentation had suggested I needed to re-state the job name (DeployInfrastructure) as the variable name prefix, but actually I needed the work Deploy instead. Possibly I have misunderstood, but anyway the above trick should avoid any trial and error in future!

(Note that for cross-stage dependencies you can use the same inspection trick, but use the stagedependencies object instead of dependencies).