Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get all values of an attribute of json array with jsonpath bigquery in bigquery? Asterisk operator not supported.

I'm trying to get all the values of a certain attribute from a json array. Considering the following json, I'm trying to get all the types e.g. iPhone,home

{
 "firstName": "John",
 "lastName" : "doe",
 "age"      : 26,
 "address"  :
 {
     "streetAddress": "naist street",
     "city"         : "Nara",
     "postalCode"   : "630-0192"
 },
 "phoneNumbers":
 [
     {
       "type"  : "iPhone",
       "number": "0123-4567-8888"
     },
     {
       "type"  : "home",
       "number": "0123-4567-8910"
     }
 ]
}

I am using $.phoneNumbers[*].type which seems to work fine on online parsers but when I'm using it in big query:

select json_extract(my_column,'$.phoneNumbers[*].type')
from my_table

I get:

JSONPath parse error at: [*].type
like image 353
Elad Ben Akoune Avatar asked Feb 25 '15 13:02

Elad Ben Akoune


People also ask

What is JSON_ extract?

JSON_EXTRACT. Extracts a JSON value, such as an array or object, or a JSON scalar value, such as a string, number, or boolean. JSON-formatted STRING or JSON. JSON_EXTRACT_SCALAR. Extracts a scalar value.

Can BigQuery read JSON?

BigQuery supports the JSON type even if schema information is not known at the time of ingestion. A field that is declared as JSON type is loaded with the raw JSON values.


1 Answers

You can write a Javascript UDF to do the extraction:

SELECT JSON_EXTRACT('[1,2,3]', '$[*]') parsed 
Error: Unsupported operator in JSONPath: *

UDF alternative:

#standardSQL
CREATE TEMPORARY FUNCTION parseJson(libs STRING)
    RETURNS ARRAY<INT64>
    LANGUAGE js AS """
      try {
        return JSON.parse(libs);
      } catch (e) {
        return [];
      }
    """;
SELECT parseJson('[1,2,3]') parsed 

More complex example:

#standardSQL
CREATE TEMPORARY FUNCTION parseJson(libs STRING)
    RETURNS ARRAY<STRUCT<x INT64, y INT64, z INT64>>
    LANGUAGE js AS """
      try {
        return JSON.parse(libs);
      } catch (e) {
        return [];
      }
    """;
SELECT parseJson(JSON_EXTRACT('{"a":[{"x":1},{"y":2},{"z":3}]}', '$.a')) parsed 

(inspired by: https://discuss.httparchive.org/t/javascript-library-detection/955)

like image 159
Felipe Hoffa Avatar answered Sep 24 '22 01:09

Felipe Hoffa