Google Cloud Build fetch Identity token

Question

in my scenario, I would like to trigger an Google Cloud Function based on HTTP endpoint during a Google Cloud Build. The HTTP request is done using a step with a python:3.7-slim container.

Based on this and this examples from the documentation, I wanted to use the following code:

REGION = 'us-central1'
PROJECT_ID = 'name-of-project'
RECEIVING_FUNCTION = 'my-cloud-function'

function_url = f'https://{REGION}-{PROJECT_ID}.cloudfunctions.net/{RECEIVING_FUNCTION}'

metadata_server_url = 'http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience='
token_full_url = metadata_server_url + function_url
token_headers = {'Metadata-Flavor': 'Google'}

token_response = requests.get(token_full_url, headers=token_headers)
jwt = token_response.text
print(jwt)

r = requests.post(url=function_url, headers=function_headers, json=payload)

Surprisingly, the code fails because jwt is Not Found (according to the print statement). I already tested the code and IAM settings by hard coding a valid identity token and also tested the exact same fetching mechanism on a test VM inside the same project. The problem seems to be that the meta data fetching some is not working inside cloud build.

Am I missing something? Thank you for any help!

Answer 1

The solution is to use a new IAM api to generate an ID_TOKEN, on a service account with an access token, if the requester (this one who generate the access token) has the role Service Account Token Creator on the service account (or widely on the project).

This first example use direct API calls

 - name: gcr.io/cloud-builders/gcloud
   entrypoint: "bash"
   args:
    - "-c"
    - |
        curl -X POST -H "content-type: application/json" \
        -H "Authorization: Bearer $(gcloud auth print-access-token)" \
        -d '{"audience": "YOUR AUDIENCE"}' \
         "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/YOUR SERVICE ACCOUNT:generateIdToken"
        # Use Cloud Build Service Account
        # service_account_email=$(gcloud config get-value account) 

And here the Python code version

- name: python:3.7
          entrypoint: "bash"
          args:
            - "-c"
            - |
                    pip3 install google-auth requests
                    python3 extract-token.py

And extract-token.py content the following code

REGION = 'us-central1'
PROJECT_ID = 'name-of-project'
RECEIVING_FUNCTION = 'my-cloud-function'
function_url = f'https://{REGION}-{PROJECT_ID}.cloudfunctions.net/{RECEIVING_FUNCTION}'

import google.auth
credentials, project_id = google.auth.default(scopes='https://www.googleapis.com/auth/cloud-platform')

# To use the Cloud Build service account email
service_account_email = credentials.service_account_email
#service_account_email = "YOUR OWN SERVICE ACCOUNT"

metadata_server_url = f'https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/{service_account_email}:generateIdToken'
token_headers = {'content-type': 'application/json'}

from google.auth.transport.requests import AuthorizedSession
authed_session = AuthorizedSession(credentials)
import json
body = json.dumps({'audience': function_url})

token_response = authed_session.request('POST',metadata_server_url, data=body, headers=token_headers)
jwt = token_response.json()
print(jwt['token'])

Don't hesitate if you need more details.

I think I will write an article on this on Medium, if you want I name you, let me know

Answer 2

The best here is to create a Feature Request (FR) in the Public Issue Tracker. There is a difference between filing an issue and a FR. The FR gives visibility to the Engineering team of the real needs; according to the number of users which are being affected by that, they prioritize the development of them. I suggest also to create a guthub repo so they can easily replicate it and make reference to the issues aforementioned.

On the other hand as a workaround, you can create a topic in Pub/Sub to receive build notifications:

gcloud pubsub topics create cloud-builds

Each time you submit a build, a message will be pushed to the topic, then you can create a PubSub Cloud Function and call your HTTP CF from there.

I used this example from github, mentioned in the docs Authenticating Function to function

const {get} = require('axios');

// TODO(developer): set these values
const REGION = 'us-central1';
const PROJECT_ID = 'YOUR PROJECTID';
const RECEIVING_FUNCTION = 'FUNCTION TO TRIGGER';

// Constants for setting up metadata server request
// See https://cloud.google.com/compute/docs/instances/verifying-instance-identity#request_signature
const functionURL = `https://${REGION}-${PROJECT_ID}.cloudfunctions.net/${RECEIVING_FUNCTION}`;
const metadataServerURL =
  'http://metadata/computeMetadata/v1/instance/service-accounts/default/identity?audience=';
const tokenUrl = metadataServerURL + functionURL;

exports.helloPubSub = async (event, context) => {
// Fetch the token
  const message = event.data
    ? Buffer.from(event.data, 'base64').toString()
    : 'Hello, World';
    
  const tokenResponse = await get(tokenUrl, {
    headers: {
      'Metadata-Flavor': 'Google',
    },
  });
  const token = tokenResponse.data;

  // Provide the token in the request to the receiving function
  try {
    const functionResponse = await get(functionURL, {
      headers: {Authorization: `bearer ${token}`},
    });
    console.log(message);
  } catch (err) {
    console.error(err);
  }
};

Finally, when the ClouBuild submits, your PubSub CF will be triggered and you can call your CF inside it.