Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Export results from BigQuery to file using bq query command

Is there a way to export the results from BigQuery to a csv file using

bq query "SELECT name,count FROM mydataset.babynames WHERE gender = 'M' ORDER BY count DESC LIMIT 6" command.

I found that we can give --destination_table=mydataset.happyhalloween parameter which will write to a different table. Is there a similar way to write it to a file?

I also tried bq query "SELECT name,count FROM mydataset.babynames WHERE gender = 'M' ORDER BY count DESC LIMIT 6" > output.txt

But this creates additional headers enter image description here

But I only want results to be written a file

Based on an answer on this thread, I tried the following query, bq query --format=csv "SELECT commit FROM [bigquery-public-data:github_repos.commits] LIMIT 10" > output.txt It was better but I still got some unnecessary text in the file output.txt

enter image description here

like image 997
abhishek jha Avatar asked Aug 16 '16 09:08

abhishek jha


1 Answers

Useful common flags

Common flags are used between bq and the command. For a full list of flags, call bq --help.

Here are some of the most useful flags:

--apilog - Turn on logging of all server requests and responses. If no string is provided (--apilog=), log to stdout; if a string is provided, instead log to that file (--apilog=filename).

--format [none|json|prettyjson|csv|sparse|pretty] - The output format.
like image 58
Pentium10 Avatar answered Sep 21 '22 12:09

Pentium10