WRITE_TRUNCATE behaviour in Big Query

Question

I have a question around the WRITE_TRUNCATE behaviour in Big Query.

I have a big query table (T1) which I'm periodically appending to with log data (one row per log line). I want to have a dataflow job (D1) that reads from this table, removes any duplicate rows and performs other data cleansing operations and then outputs this to another big query table (T2), replacing any data that may have already been present in this table. I believe I can do this by using the WRITE_TRUNCATE write disposition in the BigQuery.IO sink within the dataflow job.

Question is, if I have another dataflow job (D2) reading from table T2 while job D1 is in the middle of a write truncate to this table, what data does D2 see, i.e. does it see the table in either the state it was in before the truncate or after the truncate has finished. Or can it see the table during any step during the truncate (e.g. part way through appending the new data)?

The javadoc linked above suggests that the truncate may not be atomic while the REST documentation for Big Query suggests that it is.

Answer 1

The REST API is actually the source of truth here, i.e. the change is atomic upon the BigQuery job's successful completion.