Dynamically retrieve GitHub Actions secret

Question

I'm trying to dynamically pull back a GitHub secret using GitHub Actions at runtime:

Let's say I have two GitHub Secrets:

1. SECRET_ORANGES : "This is an orange secret"
2. SECRET_APPLES : "This is an apple secret"

In my GitHub Action, I have another env variable which will differ between branches

env:
  FRUIT_NAME: APPLES

Essentially I want to find a way to do some sort of variable substitution to get the correct secret. So in one of my child jobs, I want to do something like:

env:
  FRUIT_SECRET: {{ 'SECRET_' + env.FRUIT_NAME }}

I've tried the following approaches with no luck:

secrets['SECRET_$FRUIT_NAME'] }}

I even tried a simpler approach without concatenation just to try and get it working

secrets['$FRUIT_NAME'] }}

and

{{ secrets.$FRUIT_NAME }}

None of the above worked.

Apologies if I have not explained this very well. I tried to keep my example as simple as possible.

Anyone have any idea of how to achieve this?

Alternatively, what I am trying to do is to store secrets on a per-branch basis

For example:

In customer1 code branch: SECRET_CREDENTIAL="abc123"

In customer2 code branch: SECRET_CREDENTIAL="def456"

Then I can access the correct value for SECRET_CREDENTIAL depending on which branch I am in.

Thanks!

Update: I'm getting a bit closer to what I am trying to achieve:

name: Test

env:
  CUSTOMER: CUSTOMER1

jobs:
  build:
    runs-on: ubuntu-latest
    env:
      AWS_ACCESS_KEY_ID: ${{ env.CUSTOMER }}_AWS_ACCESS_KEY_ID
    steps:
    - uses: actions/checkout@v2
    - run: |
        AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID] }}
        echo "AWS_ACCESS_KEY_ID = $AWS_ACCESS_KEY_ID"

Answer 1

There is a much cleaner option to achieve this using the format function.

Given set secrets DEV_A and TEST_A, the following two jobs will use those two secrets:

name: Secrets

on: [push]

jobs:

  dev:
    name: dev
    runs-on: ubuntu-18.04
    env:
      ENVIRONMENT: DEV
    steps:
      - run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}

  test:
    name: test
    runs-on: ubuntu-18.04
    env:
      ENVIRONMENT: TEST
    steps:
      - run: echo ${{ secrets[format('{0}_A', env.ENVIRONMENT)] }}

This also works with input provided through manual workflows (the workflow_dispatch event):

name: Secrets

on:
  workflow_dispatch:
    inputs:
      env:
        description: "Environment to deploy to"
        required: true

jobs:
  secrets:
    name: secrets
    runs-on: ubuntu-18.04
    steps:
      - run: echo ${{ secrets[format('{0}_A', github.event.inputs.env)] }}

Answer 2

Update - July 2021

I found a better way to prepare dynamic secrets in a job, and then consume those secrets as environment variables in other jobs.

Here's how it looks like in GitHub Actions.

My assumption is that each secret should be fetched according to the branch name. I'm getting the branch's name with this action rlespinasse/github-slug-action.

Go through the inline comments to understand how it all works together.

name: Dynamic Secret Names

# Assumption:
# You've created the following GitHub secrets in your repository:
# AWS_ACCESS_KEY_ID_master
# AWS_SECRET_ACCESS_KEY_master

on:
  push:

env:
  AWS_REGION: "eu-west-1"

jobs:
  prepare:
    name: Prepare
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - name: Inject slug/short variables
        uses: rlespinasse/[email protected]
      - name: Prepare Outputs
        id: prepare-step
        # Sets this step's outputs, that later on will be exported as the job's outputs
        run: |
          echo "::set-output name=aws_access_key_id_name::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}";
          echo "::set-output name=aws_secret_access_key_name::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}";
    # Sets this job's, that will be consumed by other jobs
    # https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idoutputs
    outputs:
      aws_access_key_id_name: ${{ steps.prepare-step.outputs.aws_access_key_id_name }}
      aws_secret_access_key_name: ${{ steps.prepare-step.outputs.aws_secret_access_key_name }}

  test:
    name: Test
    # Must wait for `prepare` to complete so it can use `${{ needs.prepare.outputs.{output_name} }}`
    # https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#needs-context
    needs:
      - prepare
    runs-on: ubuntu-20.04
    env:
      # Get secret names
      AWS_ACCESS_KEY_ID_NAME: ${{ needs.prepare.outputs.aws_access_key_id_name }}
      AWS_SECRET_ACCESS_KEY_NAME: ${{ needs.prepare.outputs.aws_secret_access_key_name }}
    steps:
      - uses: actions/checkout@v2
      - name: Test Application
        env:
          # Inject secret values to environment variables
          AWS_ACCESS_KEY_ID: ${{ secrets[env.AWS_ACCESS_KEY_ID_NAME] }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets[env.AWS_SECRET_ACCESS_KEY_NAME] }}
        run: |
          printenv | grep AWS_
          aws s3 ls

Update - August 2020

Following some hands-on experience with this project terraform-monorepo, here's an example of how I managed to use secret names dynamically

1. Secrets names are aligned with environments names and branches names - development, staging and production
2. $GITHUB_REF_SLUG comes from the Slug GitHub Action which fetches the name of the branch
3. The commands which perform the parsing are

      - name: set-aws-credentials
        run: |
          echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
          echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
      - name: terraform-apply
        run: |
          export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
          export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}

Full example

name: pipeline

on:
  push:
    branches: [development, staging, production]
    paths-ignore:
      - "README.md"

jobs:
  terraform:
    runs-on: ubuntu-latest

    env:
      ### -----------------------
      ### Available in all steps, change app_name to your app_name
      TF_VAR_app_name: tfmonorepo
      ### -----------------------

    steps:
      - uses: actions/checkout@v2
      - name: Inject slug/short variables
        uses: rlespinasse/[email protected]
      - name: prepare-files-folders
        run: |
          mkdir -p ${GITHUB_REF_SLUG}/
          cp live/*.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/
          cp live/*.tf ${GITHUB_REF_SLUG}/
          cp live/*.tpl ${GITHUB_REF_SLUG}/ 2>/dev/null || true
          mv ${GITHUB_REF_SLUG}/backend.tf.${GITHUB_REF_SLUG} ${GITHUB_REF_SLUG}/backend.tf
      - name: install-terraform
        uses: little-core-labs/install-terraform@v1
        with:
          version: 0.12.28
      - name: set-aws-credentials
        run: |
          echo "::set-env name=AWS_ACCESS_KEY_ID_SECRET_NAME::AWS_ACCESS_KEY_ID_${GITHUB_REF_SLUG}"
          echo "::set-env name=AWS_SECRET_ACCESS_KEY_SECRET_NAME::AWS_SECRET_ACCESS_KEY_${GITHUB_REF_SLUG}"
      - name: terraform-apply
        run: |
          export AWS_ACCESS_KEY_ID=${{ secrets[env.AWS_ACCESS_KEY_ID_SECRET_NAME] }}
          export AWS_SECRET_ACCESS_KEY=${{ secrets[env.AWS_SECRET_ACCESS_KEY_SECRET_NAME] }}
          cd ${GITHUB_REF_SLUG}/
          terraform version
          rm -rf .terraform
          terraform init -input=false
          terraform get
          terraform validate
          terraform plan -out=plan.tfout -var environment=${GITHUB_REF_SLUG}
          terraform apply -auto-approve plan.tfout 
          rm -rf .terraform

---

After reading this - Context and expression syntax for GitHub Actions , focusing on env object, I found out that:

As part of an expression, you may access context information using one of two syntaxes.

Index syntax: github['sha']

Property dereference syntax: github.sha

So the same behavior applies to secrets, you can do secrets[secret_name], so you can do the following

    - name: Run a multi-line script
      env:
        SECRET_NAME: A_FRUIT_NAME
      run: |
        echo "SECRET_NAME = $SECRET_NAME"
        echo "SECRET_NAME = ${{ env.SECRET_NAME }}"
        SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}
        echo "SECRET_VALUE = $SECRET_VALUE"

Which results in

SECRET_NAME = A_FRUIT_NAME
SECRET_NAME = A_FRUIT_NAME
SECRET_VALUE = ***

Since the SECRET_VALUE is redacted, we can assume that the real secret was fetched.

Things that I learned -

1. You can't reference env from another env, so this won't work

   env:
     SECRET_PREFIX: A
     SECRET_NAME: ${{ env.SECRET_PREFIX }}_FRUIT_NAME
   

   The result of SECRET_NAME is _FRUIT_NAME, not good

2. You can use context expressions in your code, not only in env, you can see that in SECRET_VALUE=${{ secrets[env.SECRET_NAME] }}, which is cool

And of course - here's the workflow that I tested - https://github.com/unfor19/gha-play/runs/595345435?check_suite_focus=true - check the Run a multi-line script step