How to invoke Cloud Function from Cloud Scheduler with Authentication

Question

I've looked everywhere and it seems people either use pubsub, app engine http or http with no auth. Not too many people out there showing their work for accessing functions via authentication w/ oidc tokens to access google functions.

I checked out: Cannot invoke Google Cloud Function from GCP Scheduler but nothing seemed to work.

Documentation I followed: https://cloud.google.com/scheduler/docs/http-target-auth#using-gcloud_1

1. created a new service account
2. set roles (Cloud scheduler service agent/Cloud functions service agent/Cloud scheduler admin/cloud functions invoker...even tried owner!)
3. deployed google function that doesn't allow public (unauthenticated) access (a simple helloworld function)
4. setup cron job on cloud scheduler to run every minute against the new deployed function with this configuration:
   • url = helloworld function
   • oidc-token
   • newly created service account
   • audience set to hello world function url

outcome on cloud scheduler logs:

Expand all | Collapse all{
 httpRequest: {
 }
 insertId: "ibboa4fg7l1s9"  
 jsonPayload: {
  @type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"   
  jobName: "projects/project/locations/region/jobs/tester"   
  status: "PERMISSION_DENIED"   
  targetType: "HTTP"   
  url: "https://region-project.cloudfunctions.net/tester"   
 }
 logName: "projects/project/logs/cloudscheduler.googleapis.com%2Fexecutions"  
 receiveTimestamp: "2020-04-15T17:50:14.287689800Z"  
 resource: {…}  
 severity: "ERROR"  
 timestamp: "2020-04-15T17:50:14.287689800Z" 

I saw one solution that showed someone creating a new project to get to this to work, are there any others??

Appreciate any help provided.

UPDATE

New Google Function - running in central (same as my app engine app)

New Service Account - w/ Owner role

New Scheduled Task - Info

New Scheduled Task - Status

New Scheduled Task - Logs

ACTUAL FIX

If you're missing the cloudscheduler service account (ex: service-1231231231412@gcp-sa-cloudscheduler.iam.gserviceaccount.com) Http auth tasks wont work. To fix, I had to disable api and renable and it gave me the service account, I didnt use this service account but, that was the only changing factor after I did this to make it work.

Answer 1

These are the exact steps you have to take. Be sure not to skip the second step, it sets invoker permissions on the service account so that the scheduler is able to invoke the HTTP Cloud Function with that service account's OIDC information. Note: for simplicity, I choose the default service account here, however, it would be wise to create a separate service account for this purpose with less privileges.

# Create cloud function
gcloud functions deploy my_function \
  --entry-point=my_entrypoint \
  --runtime=python37 \
  --trigger-http \
  --region=europe-west1 \
  --project=${PROJECT_ID}

# Set invoke permissions
gcloud functions add-iam-policy-binding my_function \
  --region=europe-west1 \
  --member=serviceAccount:${PROJECT_ID}@appspot.gserviceaccount.com \
  --role="roles/cloudfunctions.invoker" \
  --project=${PROJECT_ID}

# Deploy scheduler
gcloud scheduler jobs create http my_job \
  --schedule="every 60 minutes" \
  --uri="https://europe-west1-${PROJECT_ID}.cloudfunctions.net/my_function/" \
  --http-method=POST \
  --oidc-service-account-email="${PROJECT_ID}@appspot.gserviceaccount.com" \
  --oidc-token-audience="https://europe-west1-${PROJECT_ID}.cloudfunctions.net/my_function" \
  --project=${PROJECT_ID}

Answer 2

I thought I'd expand on Nebulastic's answer as I just discovered there are a few edge cases where their answer isn't complete.

Create Cloud Function

# Create cloud function
gcloud functions deploy my_function \
  --entry-point=my_entrypoint \
  --runtime=python37 \
  --trigger-http \
  --region=europe-west1 \
  --project=${PROJECT_ID}

You may want to change fields such as entry-point, runtime or trigger. Read more in the Cloud Functions deploy docs.

Create Cloud Scheduler job:

# Deploy scheduler
gcloud scheduler jobs create http my_job \
  --schedule="every 60 minutes" \
  --uri="https://europe-west1-${PROJECT_ID}.cloudfunctions.net/my_function/" \
  --http-method=POST \
  --oidc-service-account-email="${SERVICE_ACCOUNT_EMAIL}" \
  --oidc-token-audience="https://europe-west1-${PROJECT_ID}.cloudfunctions.net/my_function" \
  --project=${PROJECT_ID}

See Cloud Schedule docs for more.

Note that any service account can be used for the OIDC authentication, not only the default one created for the project.

Furthermore, you don't need to call gcloud functions add-iam-policy-binding like suggested in Nebulastic's answer, but instead you can set the Cloud Functions Invoker role to the service account passed to --oidc-service-account-email (see doc on adding roles). This way such service account would have permissions to call any Cloud Functions without needing to grant such permission on each deployment. This isn't to say the method Nebulastic suggests is incorrect, you can also do it that way.

You can verify which service accounts have permission to call the function by clicking on its name in the Cloud Functions list -> Permissions tab -> Roles subtab -> open Cloud Functions Invoker row.

---

Now for the edge cases:

Ingress - Allow internal traffic only

Despite what's suggested by the documentation, setting function's ingress settings to 'Allow internal traffic only' does not encompass Cloud Scheduler's traffic and will result in PERMISSION_DENIED error. This is something GCP devs are aware of and it may be fixed in the future. For now, use 'Allow all traffic' (or --ingress-settings=all if deploying using gcloud).

URL parameters and OIDC authentication

If your Cloud Schedule job uses OIDC authentication and your function expects arguments in form of URL parameters - eg. ...my_function?key=value - then you must ensure the OIDC Audience does not contain the URL parameters. Specify the URL parameters in the URL field only, but if they appear in Audience field, the request will return 403 UNAUTHENTICATED. Note that OIDC Audience is auto-filled in with a copy of the URL if you don't specify the OIDC Audience manually. This is another issue GCP devs know about and are potentially working on a fix or an update to the documentation.

tldr: don't put URL parameters in OIDC audience URL - only in the URL field:

  --uri="https://europe-west1-${PROJECT_ID}.cloudfunctions.net/my_function?key=value" \
  --oidc-token-audience="https://europe-west1-${PROJECT_ID}.cloudfunctions.net/my_function" \

JSON parameters and Content-Type

If you choose to use a Cloud Scheduler job calling a function with JSON parameters - eg. {"key": "value"} in the body field - then you will need to either:

• Parse the JSON manually in the function - for instance Flask's get_json won't work unless you call it with force=True.

• Create the function using gcloud scheduler jobs create, rather than from the Console, and ensure you specify the Content-Type header by adding the following flag: --headers Content-Type=application/json

Cloud Scheduler API enabled before 19/03/2019

The Cloud Scheduler service account with this role granted is automatically set up when you enable the Cloud Scheduler API, unless you enabled it prior to March 19, 2019, in which case you must add the role manually.

This seems to be the reason behind the issue listed in the author's question. See docs for more. The reason this caused issues is as follows:

Cloud Scheduler itself must have a service account of its own that has the Cloud Scheduler Service Agent role granted. This is so it can generate header tokens on behalf of your client service account to authenticate to your target.

---

Sources: just spent a week working with one of their support agents to figure all of this out - shout out to Jason who helped uncover all these quirks. They're working on these issues, but I thought I'd leave this info here until (if ever) these things get addressed.

Answer 3

This is a frustrating bug that has been resolved for those who activated the api after 3/19/2019.

For those before 3/19/2019 google has noted the fix below;

This process allows you to resolve the issue without disabling and re-enabling the API.

Google has a document that addresses this issue and states

"This is necessary only if you enabled Cloud Scheduler API prior to March 19, 2019."

https://cloud.google.com/scheduler/docs/http-target-auth#add

Using the console:

Using gcloud

Find your project number:

gcloud projects describe [project-id] --format='table(projectNumber)'
Replacing [project-id] with your project ID.

Copy down the number.

Grant the Cloud Scheduler service account the Cloud Scheduler Service Agent role, using the project number you copied down:

gcloud projects add-iam-policy-binding [project-id] --member serviceAccount:service-[project-number]@gcp-sa-cloudscheduler.iam.gserviceaccount.com --role roles/cloudscheduler.serviceAgent

Replacing [project-id] with your project ID and [project-number] with the project number from above.

Note from comments above: Disabling and re-enabling the API also resolves the issue (as it does this procedure automatically) but it is disruptive for anybody who doesn't want to destroy their current tasks.