I am trying to figure out the number of slots used by every big query query. Is there a way to find it out?
This information is indeed available in job.statistics.query.timeline that forms parts of the Jobs API of BigQuery (https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#resource). When you get this information it comes in an array like this:
timeline:
[ '{"elapsedMs":"750","totalSlotMs":"2795","pendingUnits":"8","completedUnits":"66","activeUnits":"9"}',
'{"elapsedMs":"1252","totalSlotMs":"3617","pendingUnits":"1","completedUnits":"73","activeUnits":"1"}',
'{"elapsedMs":"2944","totalSlotMs":"5643","pendingUnits":"0","completedUnits":"78","activeUnits":"0"}' ],
So what you can do depends on your actual question:
1) If your question is "What is the total amount of slots utilised by the query over its running time?", then look at the final value of completedUnits
2) If your question is "How are the slots utilised during the query's running time?", then you can build an average of completedUnits
over the elapsedMs
per timeslice.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With