Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

zcat on amazon s3

I am wondering if it is possible to cat a gzip file stored on amazon s3. Maybe using some streamming client. What do you think?

We are loking for a operation similar to zcat s3://bucket_name/your_file | grep "log_id"

like image 840
raimonbosch Avatar asked Oct 11 '11 16:10

raimonbosch


1 Answers

Found this thread today, and liked Keith's answer. Fast forward to today's aws cli it's done with:

aws s3 cp s3://some-bucket/some-file.bz2 - | bzcat -c | mysql -uroot some_db

Might save someone else a tiny bit of time.

like image 153
sdlarsen Avatar answered Oct 12 '22 11:10

sdlarsen