My Google Bigquery table have date and time column. New data are posted in table every 10 minutes, so date field would have e.g. "2018-10-26" and time field "19:05:00". Next record would be like "2018-10-26" and "19:15:00" for date and time field. How to aggregate data for each day by one hour (24 records per day)?
SQL request is sent from Google Sheets using Apps Script. Here is part of google bigquery.gs script: (complete script in GitHub)
...
var sheet = SpreadsheetApp.getActiveSheet();
var sql = 'SELECT date, time, SUM(col1) AS Col1, SUM(col2) AS Col2 GROUP BY
time, date ORDER BY time ASC';
var queryResults;
// Inserts a Query Job
try {
var queryRequest = BigQuery.newQueryRequest();
queryRequest.setQuery(sql).setTimeoutMs(100000);
queryResults = BigQuery.Jobs.query(queryRequest, projectNumber);
}
....
You can set the duration of the time travel window, from a minimum of two days to a maximum of seven days. Seven days is the default.
Datetime type: comprises both calendar date and time. It does not store time zone information: YYYY-MM-DD HH:MM:SS (e.g. ). Timestamp type: comprises date, time, and time zone information.
If user want to save a timestamp when a request reaching the BigQuery stream system, user could add a TIMESTAMP column in the table schema, and assign "AUTO" in your json payload. With "AUTO" key words, the system will set the column value to be the wall timestamp on behalf of the user.
Below is for BigQuery Standard SQL
#standardSQL
SELECT date, TIME_TRUNC(time, HOUR) hour, SUM(col1) AS Col1, SUM(col2) AS Col2
FROM `project.dataset.table`
GROUP BY date, hour
ORDER BY date, hour
You can test, play with above using dummy data in your question:
#standardSQL
WITH `project.dataset.table` AS (
SELECT DATE "2018-10-26" date, TIME "19:05:00" time, 1 col1, 2 col2 UNION ALL
SELECT "2018-10-26", "19:15:00", 3, 4
)
SELECT date, TIME_TRUNC(time, HOUR) hour, SUM(col1) AS Col1, SUM(col2) AS Col2
FROM `project.dataset.table`
GROUP BY date, hour
ORDER BY date, hour
with result
Row date hour Col1 Col2
1 2018-10-26 19:00:00 4 6
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With