Trigger cloud function when new data in BigQuery

Question

I would like to trigger a Cloud Function when new data has been imported into a BigQuery table. Ideally, I would like to extract all the rows (one column is ISIN) that have been inserted.

Would this be possible and how?

Answer 1

Unfortunately BigQuery has no triggers as a native feature as of Jan 2021 :/ However there is a nice workaround you can implement! :)

As you probably know, almost every operation in GCP is logged to Cloud Logging (previously StackDriver Logging). In this article they demonstrate how to trigger a Cloud Function whenever a BigQuery load job finishes. The idea is simple:

1. Create a filter for BigQuery load jobs in Cloud Logging to catch when load jobs terminate.
2. Create a sink from Cloud Logging to Pub/Sub topic.
3. Set your Cloud Function as a consumer on this topic and implement your logic