Menu
Currently, the Athena query results are in tsv format in S3. Is there any way to configure Athena queries to return results in Parquet format.
950
CSV is the only output format used by the Athena SELECT query, but you can use UNLOAD to write the output of a SELECT query to the formats that UNLOAD supports. Although you can use the CTAS statement to output data in formats other than CSV, those statements also require the creation of a table in Athena.
Amazon Athena supports a wide variety of data formats like CSV, TSV, JSON, or Textfiles and also supports open source columnar formats such as Apache ORC and Apache Parquet. Athena also supports compressed data in Snappy, Zlib, LZO, and GZIP formats.
Amazon Athena is a interactive query service that makes it easy to analyze data directly in Amazon S3 using standard SQL. Athena stores all the query result in an S3 bucket.
Answer
At this moment it isn't possible to do it directly with Athena. When it comes to configure result of the Athena query you can only setup query result location and encryption configuration.
Workaround
1) From October Athena supports CTAS query, you can try to use this feature.
https://docs.aws.amazon.com/athena/latest/ug/ctas.html
https://docs.aws.amazon.com/athena/latest/ug/ctas-examples.html
I think example 4 or 5 may interest you.
2) You can create Python script and use Boto3 framework. Here is the example, you can adapt this script to your requirements and add saving query results to the parquet format.
https://gist.github.com/schledererj/b2e2a800998d61af2bbdd1cd50e08b76
3) If you want to transform your data you should use rather such services as AWS EMR or AWS Glue.
151
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
