Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Number of columns limitation in BigQuery?

Is there limitation on number of columns in BigQuery?

I'm thinking to prepare large number of columns as suggested the best practices. https://developers.google.com/bigquery/articles/ingestionbestpractices

like image 581
kawty Avatar asked Aug 10 '12 09:08

kawty


2 Answers

The limit is 10,000 columns. This limit was imposed as a somewhat arbitrary one, with the assumption that no one would have more than 10k columns. Looking at the error logs, someone indeed hit this limitation and got an error. There is also a 64k limit on the total size of the row (bytes of CSV data) on import.

like image 145
Jordan Tigani Avatar answered Oct 16 '22 02:10

Jordan Tigani


2013 update: The row size limit has gone from 64k to 1 MB for CSV and 20 MB for JSON data. It's a good idea to keep the number of columns below 10,000.

Check https://cloud.google.com/bigquery/docs/creating-clustered-tables#limitations

like image 39
Felipe Hoffa Avatar answered Oct 16 '22 01:10

Felipe Hoffa