Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

BigQuery: insert rows, but it's not written

I am updating a list of rows using the method tabledata().insertAll from the bigquery object. After the execution, the return shows no errors. However, my tables still continue with no data written.

Could be a problem of permission. If so, why there's no errors returned?

Thanks

like image 981
Eduardo Tenório Avatar asked Mar 14 '23 08:03

Eduardo Tenório


1 Answers

This can happen if you do the insert right after deleting and re-creating the table. The streaming buffer of a deleted table is not deleted right at the time that table is deleted, which can cause new inserts to be delivered to this old streaming buffer.

From BigQuery documentation:

deleting and/or recreating a table may create a period of time where streaming inserts are effectively delivered to the old table and will not be present in the newly created table.

And in this case, no errors would be returned.


References:

  • https://cloud.google.com/bigquery/troubleshooting-errors#metadata-errors-for-streaming-inserts
  • https://github.com/GoogleCloudPlatform/google-cloud-php/issues/871#issuecomment-361339800
  • https://cloud.google.com/bigquery/streaming-data-into-bigquery
like image 161
easd Avatar answered Mar 25 '23 05:03

easd