How do I set an environment or substitution variable via a step in Google Cloud Build?

Question

Basically, when using Google Cloud Build, how do I read a value that was written in an earlier build step in subsequent steps?

Specifically, I'd like to make a custom image tag that's based on a combination of the timestamp and $SHORT_SHA. Something like the below. Though, it doesn't work, as docker complains about "export", and, even if that worked, it likely will be a different env:

  # Setting tag in a variable:
  - name: 'ubuntu'
    args: ['export', '_BUILD_TAG=`date', '-u', '+%Y%m%dT%H%M%S_$SHORT_SHA`']

Then, in a later step:

  # Using tag from the variable:
  - name: gcr.io/cloud-builders/docker
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/$_BUILD_TAG', '.']

So, how do I use the output of one step in another? I could write the contents of date to a file, and then read it, but I'm back at not knowing how to set the variable from the file I read (or otherwise interpolate its results to form the argument to docker build).

Answer 1

I never found a way to set an environment variable in one build step that can be read in other steps, but I ended up accomplishing the same effect by building on Konstantin's answer in the following way:

In an early step, I generate and write my date-based tag to a file. The filesystem (/workspace) is retained between steps, and serves as store of my environment variable. Then, in each step that I need to reference that value, I cat that file in place. The trick is to use sh or bash as the entrypoint in each container so that the sub-shell that reads from the file can execute.

Here's an example:

## Set build tag and write to file _TAG
- name: 'ubuntu'
  args: ['bash', '-c', 'date -u +%Y%m%dT%H%M_$SHORT_SHA > _TAG']

...

# Using the _TAG during Docker build:
- name: gcr.io/cloud-builders/docker
entrypoint: sh
args: ['-c', 'docker build -t gcr.io/$PROJECT_ID/image_name:$(cat _TAG) .']

A caveat to note is that if you are doing the bash interpolation in this way within, say, a JSON object or something that requires double quotes, you need the subshell call to never be surrounded by single quotes when executed in the container, only double, which may require escaping the internal double quotes to build the JSON object. Here's an example where I patch the kubernetes config using the _TAG file value to deploy the newly-build image:

- name: gcr.io/cloud-builders/kubectl
entrypoint: bash
args: ['-c', 'gcloud container clusters get-credentials --zone $$CLOUDSDK_COMPUTE_ZONE $$CLOUDSDK_CONTAINER_CLUSTER ; kubectl patch deployment deployment_name -n mynamespace -p "{\"spec\":{\"template\":{\"spec\":{\"containers\":[{\"name\":\"image_name\",\"image\":\"gcr.io/$PROJECT_ID/image_name:$(cat _TAG)\"}]}}}}}"']
env:
- 'CLOUDSDK_COMPUTE_ZONE=us-central1-b'
- 'CLOUDSDK_CONTAINER_CLUSTER=my-google-proj-cluster-name'

Answer 2

- name: gcr.io/cloud-builders/docker
  entrypoint: sh
  args
    - '-c'
    - 'docker build -t gcr.io/$PROJECT_ID/$(date -u +%Y%m%dT%H%M%S_$SHORT_SHA) .'

Answer 3

Although this doesn't solve your problem, I did want to post this answer since the first sentence of your question is, "Basically, when using Google Cloud Build, how do I read a value that was written in an earlier build step in subsequent steps?". This is how you'd do that.

From the official documentation:

A Volume is a Docker container that is mounted into build steps to persist files across build steps. When Cloud Build runs a build step, it automatically mounts a workspace volume into /workspace. You can specify additional volumes to be mounted into your build steps' containers using the volumes field for your steps.

Here's an example of it implemented from someone who asked this question in a github issue, but to put the date in the volume for later reading by another step:

steps:
- name: 'ubuntu'
  volumes:
  - name: 'vol1'
    path: '/persistent_volume'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
        date -u +%Y%m%dT%H%M_$SHORT_SHA > /persistent_volume/file
- name: 'gcr.io/cloud-builders/docker'
  volumes:
  - name: 'vol1'
    path: '/persistent_volume'
  args: ['run', '-v', 'vol1:/data', 'alpine', 'cat', 'data/file']

However, for your particular case, I would just tag it with a subshell command like it's done in this answer here:

$(date -u +%Y%m%dT%H%M%S_$SHORT_SHA)