Is it possible to add a new field to an existing field of RECORD type in bigquery? So for example if my current schema is :
{u'fields': [{u'mode': u'NULLABLE', u'name': u'test1', u'type': u'STRING'},
{u'fields': [{u'mode': u'NULLABLE',
u'name': u'field1',
u'type': u'STRING'}],
u'mode': u'NULLABLE',
u'name': u'recordtest',
u'type': u'RECORD'}]}
Can I change it to add field "field2" to recordtest? So the new schema will look like:
{u'fields': [{u'mode': u'NULLABLE', u'name': u'test1', u'type': u'STRING'},
{u'fields': [{u'mode': u'NULLABLE',
u'name': u'field1',
u'type': u'STRING'},
{u'mode': u'NULLABLE',
u'name': u'field2',
u'type': u'STRING'}],
u'mode': u'NULLABLE',
u'name': u'recordtest',
u'type': u'RECORD'}]}
I couldn't find a way to do this from the UI. But I was able to do this using the API.
The BigQuery data manipulation language (DML) enables you to update, insert, and delete data from your BigQuery tables. You can execute DML statements just as you would a SELECT statement, with the following conditions: You must use Google Standard SQL. To enable Google Standard SQL, see Switching SQL dialects.
You can modify the data type in a table with the ALTER COLUMN SET DATA TYPE statement in BigQuery. For example, an INT64 data type can be changed into a FLOAT64 type, but not the other way around.
You can also use the bigquery CLI tool which comes as part of the gcloud command line tool. I found this route easier because there is less information to add to the request. You don't need an access token or API key since your session is already established. You can find everything you need to know here but the basics of it is:
bq show \ --schema \ --format=prettyjson \ YOUR_PROJECT_ID:YOUR_DATASET.YOUR_TABLE_NAME > schema_file.json
Then modify the schema file to add the new nested field.
Then run:
bq update YOUR_PROJECT_ID:YOUR_DATASET.YOUR_TABLE_NAME schema_file.json
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With