Menu
I have an S3 bucket with several zipped CSV files (utilization logs.) I'd like to query this data with Athena, but the output is completely garbled.
It appears Athena is trying to parse the zip files without decompressing them first. Is it possible to force Hive to recognize my files as compressed data?
676
Athena supports a variety of compression formats for reading and writing data, including reading from a table that uses multiple compression formats.
So Whats the Difference Between S3 Select and Athena? S3 Select is a lightweight solution designed to let you use SQL to perform simple SELECT clauses on a maximum of one file. Amazon Athena is an analytics workhorse that allows you to perform SQL on extremely large datasets spanning many files with great performance.
S3 does not support stream compression nor is it possible to compress the uploaded file remotely.
The new Athena encrypted data feature also supports encrypting query results and storing these results in Amazon S3.
For Athena compression is supported, but the supported formats are
Those formats are detected by their filename suffix. If the suffix doesn't match, the reader does not decode the content. I tested it with a test.csv.gz file and it worked right away. So try changing the compression from zip to gzip and it should work.
144
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
