How to deploy Cloud Functions with secrets from Secret Manager using Cloud Build?

Question

I have a Cloud Function that I want deployed in my CD pipeline using Cloud Build. The function needs a couple of secrets stored in Secret Manager that I want to pull in as environment variables using the --set-secrets flag.

When I deploy manually with the CLI I have no issue:

gcloud beta functions deploy myfunction \
  --source src \
  --trigger-topic mytopic \
  --region europe-west1 \
  --runtime python39 \
  --set-secrets 'env_1=secret_1:latest','env_2=secret_2:latest'

However, when I try to deploy using Cloud Build with this configuration:

steps:
- name: 'gcr.io/cloud-builders/gcloud'
  args:
  - beta
  - functions
  - deploy
  - myfunction
  - --source=src
  - --trigger-topic=mytopic
  - --region=europe-west1
  - --runtime=python39
  - --set-secrets='env_1=secret_1:latest','env_2=secret_2:latest'

I get an error that the --set-secrets argument must match the pattern 'SECRET:VERSION' or 'projects/{PROJECT}/secrets/{SECRET}:{VERSION}' or 'projects/{PROJECT}/secrets/{SECRET}/versions/{VERSION}' where VERSION is a number or the label 'latest'. I don't understand why I get this error as I think my argument comforms to said pattern.

Is there something I am missing?

Answer 1

First, follow Guillaume's suggestion to remove the quotation marks around each pair. Afterwards, it should look like this:

--set-secrets=env_1=secret_1:latest,env_2=secret_2:latest

Or alternatively, my suggestion is to enclose all your arguments as a list like the example below. I tested the config below and it worked on my end.

steps:
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  args: ['gcloud', 'beta','functions', 'deploy', 'myfunction', '--region=europe-west1', '--source=src', '--trigger-topic=mytopic', '--runtime=python39', '--set-secrets=env_1=secret_1:latest,env_2=secret_2:latest']

Note: Do not put spaces in --set-secrets value if you have multiple secrets

To learn more, check out this documentation.

Answer 2

Here is some documentation: https://cloud.google.com/build/docs/securing-builds/use-secrets

You need to use the secretEnv key as well as the availableSecrets declaration in your cloudbuild.yaml