I'm querying BigQuery via google-cloud-python
as follows:
client = bigquery.Client()
query = """SELECT * FROM `{dataset}.{table}`
WHERE id=@id LIMIT 1""".format(dataset=dataset,
table=table)
param = ScalarQueryParameter('id', 'STRING', id)
query = client.run_sync_query(query, query_parameters=[param])
query.use_legacy_sql = False
query.timeout_ms = 1000
query.run()
assert query.complete
try:
results = query.rows[0]
except IndexError:
results = None
This returns data like:
[
"Tue, 11 Apr 2017 03:18:52 GMT",
"A132",
"United Kingdom",
[
{
"endDate": "2012-12-05",
"startDate": "2011-12-27",
"statusCode": "Terminated"
}
]
]
The repeated field has been converted to JSON. But I want the rest of the data to be converted to JSON as well. I could implement this myself by examining query.schema
but it seems like this should be in the library since it already happens for repeated elements.
How can I get BigQuery query results formatted as JSON using this library? E.g.:
{
"timestamp": "Tue, 11 Apr 2017 03:18:52 GMT",
"id": "A132",
"country": "United Kingdom",
[
{
"endDate": "2012-12-05",
"startDate": "2011-12-27",
"statusCode": "Terminated"
}
]
}
JSON_EXTRACT. Extracts a JSON value, such as an array or object, or a JSON scalar value, such as a string, number, or boolean. If a JSON key uses invalid JSONPath characters, then you can escape those characters using single quotes and brackets. Extracts a JSON null when a JSON null is encountered.
Open the BigQuery page in the Google Cloud console. In the Explorer panel, expand your project and dataset, then select the table. In the details panel, click Export and select Export to Cloud Storage.
BigQuery natively supports JSON data using the JSON data type. This document describes how to create a table with a JSON column, insert JSON data into a BigQuery table, and query JSON data.
As it turns out, the code is simple enough:
field_names = [f.name for f in query.schema]
try:
raw_results = query.rows[0]
zipped_results = zip(field_names, raw_results)
results = {x[0]: x[1] for x in zipped_results}
except IndexError:
results = None
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With