I'm trying to load a comma separated test file, 'Davetest.csv', and use a JSON schema held in the file 'testjson.json'
However I can't get it to recognize the table schema parameter properly.
load mydataset.testy gs://paps1/davetest.csv gs://paps1/testjson.json
gives me
BigQuery error in load operation: Invalid value for: //PAPS1/TESTJSON.JSON is not a valid value
I tried it without the path for the table schema (i.e.load mydataset.testy gs://paps1/davetest.csv testjson.json), and this confused it into thinking I was trying to write the schema inline with a first fieldname of 'testjson.json', and so failed on that as well.
Both of the files are held on GCS in the same bucket.
Obviously I'm doing something fundamentally wrong, but all the code examples I've seen are using the structure above, as far as I can tell.
Could anyone advise?
I think documentation for "bq Command-Line Tool" is not clear enough! https://cloud.google.com/bigquery/bq-command-line-tool#creatingtablefromfile
Rather, check examples by running > bq help load
They are more specific in showing allowed options for schema:
Examples:
bq load ds.new_tbl ./info.csv ./info_schema.json
bq load ds.new_tbl gs://mybucket/info.csv ./info_schema.json
bq load ds.small gs://mybucket/small.csv name:integer,value:string
bq load ds.small gs://mybucket/small.csv field1,field2,field3
As you can see now, schema file is expected to be local to bq tool
And, of course using inline text schema is still option if local file is somehow doesnt fit in your design
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With