BigQuery execution time

Question

After I send a query to be executed in BigQuery, how can I find the execution time and other statistics about this job?

The Python API for BigQuery has a suggestive field timeline, but barely mentioned in the documentation. However, this method returns an empty iterable.

So far, I have been running this Python code, with the BigQuery Python API.

from google.cloud import bigquery

bq_client = bigquery.Client()
job = bq_client.query(some_sql_query, location="US")
ts = list(job.timeline)
if len(ts) > 0:
    for t in ts:
        print(t)
else:
    print('No timeline results ???')    # <-- no timeline results.

Answer 1

When your job is "DONE", the BigQuery API returns a JobStatistics object (described here) that is accessible in Python through the attributes of your job object.

The available attrs are accessible here.

For accessing the time taken by your job you mainly have the job.created, job.started and job.ended attributes.

To come back to your code snippet, you can try something like this:

from google.cloud import bigquery

bq_client = bigquery.client()
job = bq_client.query(some_sql_query, location="US")

while job.running():
    print("Running..")
    sleep(0.1)

print("The job duration was {}".format(job.ended - job.started))