Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can I safely query a BigQuery table being replaced with WRITE_TRUNCATE

If configuration.load.writeDisposition is set to WRITE_TRUNCATE during a load job, is there a period of time when querying the table would raise an error?

  • The whole period when the job is marked as PENDING and/or RUNNING?
  • A small moment when the table is replaced at the end of the load job?

What would be the error? status.errors[].reason => "notFound"?

like image 453
nfo Avatar asked Jan 31 '13 13:01

nfo


People also ask

How do you overwrite a table in BigQuery?

To append to or overwrite a table using query results, specify a destination table and set the write disposition to either: Append to table — Appends the query results to an existing table. Overwrite table — Overwrites an existing table with the same name using the query results.

How do I copy a table from one project to another in BigQuery?

In the Google Cloud console, go to the BigQuery page. Click Data transfers. Select a transfer for which you want to view the transfer details. On the Transfer details page, select a transfer run.

Can you rename a BigQuery table?

There is no option to rename a table in BigQuery 😬. You can copy a table in BigQuery, though. This doesn't incur any additional charges other than the additional cost of storage. Once copied, you can delete the original table so you only get charged for the storage once.


1 Answers

The WRITE_TRUNCATE is atomic and gets applied at the end of the load job. So any queries that happen during that time will see either only the old data or all of the new data. There should be no cases where you'd get an error querying the table.

If the load failed, then there should be no change to the table, and if it succeeded, all of the data should appear at once in the table.

If the table didn't already exist, and a load job specified CREATE_IF_NEEDED, then querying the table would give not found until the load job completed.

We're working on a doc rewrite that will make this more clear.

like image 186
Jordan Tigani Avatar answered Sep 28 '22 09:09

Jordan Tigani