Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to load .zip files into BigQuery?

Tags:

We are able to load uncompressed CSV files and gzipped files completely fine. However, if we want to load CSV files compressed in ".zip" - what is the best approach to move ahead? Will we need to manually convert the zip to gz or BigQuery has added some support to handle this?

Thanks

like image 616
Dragon Avatar asked Sep 12 '18 16:09

Dragon


1 Answers

BigQuery supports loading gzip files

The limitation is - If you use gzip compression BigQuery cannot read the data in parallel. Loading compressed CSV data into BigQuery is slower than loading uncompressed data.

like image 191
Mikhail Berlyant Avatar answered Oct 06 '22 17:10

Mikhail Berlyant