Schedule query in BigQuery

Question

I have a BigQuery query which unions two tables (daily Google Analytics exports). What I want is to save the query as a view or table but have that final view/table update automatically everyday as the new Google Analytics data is loaded into BigQuery.

Thanks for your help

Answer 1

Update

Scheduling recurring queries is available now in BigQuery since September 2018

---

BigQuery by itself doesn’t provide any scheduling mechanism.

You can use 3rd party solution or develop your own using client of your choice
https://cloud.google.com/bigquery/docs/reference/libraries

For example,

You can use App Engine Cron Service to automate BigQuery query execution https://cloud.google.com/appengine/docs/java/config/cron https://cloud.google.com/appengine/features/#cron https://cloud.google.com/bigquery/bigquery-api-quickstart

Or, you can use Google Apps Script to script BigQuery query with a Time based trigger to execute that script
https://developers.google.com/apps-script/advanced/bigquery https://developers.google.com/apps-script/guides/triggers/installable#time-driven_triggers

Also, check recently introduced Google Cloud Functions (Beta)
https://cloud.google.com/functions/docs/

Answer 2

New Feature: https://cloud.google.com/bigquery/docs/scheduling-queries

BigQuery has scheduled queries feature in Beta now.

You would need to enable the BigQuery Data Transfer Service and the "scheduled queries" is in the BigQuery Transfer Service menu.