How can I trigger an email or other notification based on a BigQuery query?

Question

I would like to receive a notification, ideally via email, when some threshold is met in Google BigQuery. For example, if the query is:

SELECT name, count(id) FROM terrible_things
WHERE date(terrible_thing) < -1d

Then I would want to get an alert when there were greater than 0 results, and I would want that alert to contain the name of each object and how many there were.

Answer 1

BigQuery does not provide the kinds of services you'd need to build this without involving other technologies. However, you should be able to use something like appengine (which does have a task scheduling mechanism) to periodically issue your monitoring query probe, check the results of the job, and alert if there are nonzero rows in the results. Alternately, you could do this locally using some scripting and leveraging the BQ command line tool.

You could also refine things by using BQ's table decorators to only scan the data that's arrived since you last ran your monitoring query, if you retain knowledge of the last probe's execution in the calling system.

In short: Something else needs to issue the queries and react based on the outcome, but BQ can certainly evaluate the data.