How to do group by on repeated field in BigQuery

Question

In BigQuery, I have created a table with the below schema

id                  INTEGER NULLABLE    
visits              INTEGER NULLABLE    
dimensions          RECORD  REPEATED    
dimensions.value    STRING  
dimensions.key      STRING  

How to get sum(visits) by grouping device and state values?

Example data:

{"id": 1, visits: 100, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"CA"}]}
{"id": 1, visits: 500, "dimensions": [{"key":"device","value":"desktop"}, {"key":"state","value":"CA"}]}
{"id": 1, visits: 200, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"NY"}]}
{"id": 2, visits: 100, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"CA"}]}
{"id": 2, visits: 500, "dimensions": [{"key":"device","value":"desktop"}, {"key":"state","value":"CA"}]}
{"id": 2, visits: 200, "dimensions": [{"key":"device","value":"mobile"}, {"key":"state","value":"NY"}]}
{"id": 2, visits: 780, "dimensions": [{"key":"device","value":"desktop"}, {"key":"state","value":"NY"}]}

I want id, device, state, sum(visits) in the output.

I could do a group by using a single dimension with below query but do not know how to do it for multiple dimensions.

SELECT id,d.value, sum(visits) FROM dataset.tabe_name,UNNEST(dimensions) as d where d.key = "device" group by id, d.value LIMIT 1000

And also is it possible to write a generic query when key values are not known in advance?

Answer 1

Below is for BigQuery Standard SQL

#standardSQL
SELECT 
  id,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "device") AS device,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "state") AS state,
  SUM(visits) AS visits
FROM `dataset.tabe_name`  
GROUP BY id, device, state
LIMIT 1000   

You can try / play it with dummy data from your example as below

#standardSQL
WITH data AS (
  SELECT 1 AS id, 100 AS visits, ARRAY<STRUCT<key STRING, value STRING>>[("device", "mobile"), ("state", "CA")] AS dimensions UNION ALL
  SELECT 1, 100, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "CA")] UNION ALL
  SELECT 1, 500, [STRUCT<key STRING, value STRING>("device", "desktop"), ("state", "CA")] UNION ALL
  SELECT 1, 200, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "NY")] UNION ALL
  SELECT 2, 100, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "CA")] UNION ALL
  SELECT 2, 500, [STRUCT<key STRING, value STRING>("device", "desktop"), ("state", "CA")] UNION ALL
  SELECT 2, 200, [STRUCT<key STRING, value STRING>("device", "mobile"), ("state", "NY")] UNION ALL
  SELECT 2, 780, [STRUCT<key STRING, value STRING>("device", "desktop"), ("state", "NY")] 
)
SELECT 
  id,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "device") AS device,
  (SELECT value FROM UNNEST(dimensions) WHERE key = "state") AS state,
  SUM(visits) AS visits
FROM data  
GROUP BY id, device, state
-- ORDER BY id, device, state