Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to group by time intervals with Google BigQuery

I have TIMESTAMP weather data recorded every 5 minutes that I want to group in 15 minute intervals. I found the floor function below that looked promising, but BQ does not support the UNIX_TIMESTAMP function

SELECT
    FLOOR(UNIX_TIMESTAMP(utc_timestamp)/(15 * 60)) AS timekey
GROUP BY
    timekey

What is the best way to do this?

like image 979
atltbone Avatar asked Mar 15 '19 22:03

atltbone


1 Answers

Below is for BigQuery Standard SQL

#standardSQL
SELECT 
  TIMESTAMP_SECONDS(15*60 * DIV(UNIX_SECONDS(utc_timestamp), 15*60)) timekey,
  AVG(metric) metric
FROM `project.dataset.table`
GROUP BY timekey

You can test, play with above using dummy data as in below example

#standardSQL
WITH `project.dataset.table` AS (
  SELECT TIMESTAMP '2019-03-15 00:00:00' utc_timestamp, 1 metric UNION ALL
  SELECT '2019-03-15 00:05:00', 2 UNION ALL
  SELECT '2019-03-15 00:10:00', 3 UNION ALL
  SELECT '2019-03-15 00:15:00', 4 UNION ALL
  SELECT '2019-03-15 00:20:00', 5 UNION ALL
  SELECT '2019-03-15 00:25:00', 6 UNION ALL
  SELECT '2019-03-15 00:30:00', 7 UNION ALL
  SELECT '2019-03-15 00:35:00', 8 UNION ALL
  SELECT '2019-03-15 00:40:00', 9 
)
SELECT 
  TIMESTAMP_SECONDS(15*60 * DIV(UNIX_SECONDS(utc_timestamp), 15*60)) timekey,
  AVG(metric) metric
FROM `project.dataset.table`
GROUP BY timekey
-- ORDER BY timekey  

with result

Row timekey                 metric   
1   2019-03-15 00:00:00 UTC 2.0  
2   2019-03-15 00:15:00 UTC 5.0  
3   2019-03-15 00:30:00 UTC 8.0   

Obviously, you can use whatever aggregation your logic requires - I used AVG() just for the sake of example

like image 72
Mikhail Berlyant Avatar answered Nov 20 '22 16:11

Mikhail Berlyant