Using Python to select data from Bigquery public dataset, after getting the result need to print it in JSON format.How to convert the result to JSON ? Thanks!
Have tried row[0]
but errors out.
try:
raw_results = query.rows[0]
zipped_results = zip(field_names, raw_results)
results = {x[0]: x[1] for x in zipped_results}
except IndexError:
results = None
# from google.cloud import bigquery
# client = bigquery.Client()
query = """
SELECT word, word_count
FROM `bigquery-public-data.samples.shakespeare`
WHERE corpus = @corpus
AND word_count >= @min_word_count
ORDER BY word_count DESC;
"""
query_params = [
bigquery.ScalarQueryParameter("corpus", "STRING", "romeoandjuliet"),
bigquery.ScalarQueryParameter("min_word_count", "INT64", 250),
]
job_config = bigquery.QueryJobConfig()
job_config.query_parameters = query_params
query_job = client.query(
query,
# Location must match that of the dataset(s) referenced in the
query.location="US",
job_config=job_config,
) # API request - starts the query
# Print the results
for row in query_job:
print("{}: \t{}".format(row.word, row.word_count))
assert query_job.state == "DONE"
Open the BigQuery page in the Google Cloud console. In the Explorer panel, expand your project and dataset, then select the table. In the details panel, click Export and select Export to Cloud Storage.
BigQuery natively supports JSON data using the JSON data type. This document describes how to create a table with a JSON column, insert JSON data into a BigQuery table, and query JSON data.
There is no current method for automatic conversion, but there is a pretty simple manual method to convert to json:
records = [dict(row) for row in query_job]
json_obj = json.dumps(str(records))
Another option is to convert using pandas:
df = query_job.to_dataframe()
json_obj = df.to_json(orient='records')
You can actually just have BigQuery produce JSON directly. Change your query like this:
query = """
SELECT TO_JSON_STRING(word, word_count) AS json
FROM `bigquery-public-data.samples.shakespeare`
WHERE corpus = @corpus
AND word_count >= @min_word_count
ORDER BY word_count DESC;
"""
Now the result will have a single column named json
with JSON-formatted output.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With