Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Uploading ZIP file to S3, use EC2 to Unzip

Tags:

I have a Codeigniter web app that is uploading many tiny files every hour to Amazon S3, which is causing my S3 request charges to shoot up real fast. One way to overcome this will be to zip up the file, upload the zip file to S3, then unzip it when it is on S3.

Can this be done using EC2? Or is there a better method to achieve this? Thank you!!

EDIT: If I were to use EC2, do I use PHP to trigger the creation of a EC2 instance, upload the PHP file required to unzip the zipped files, copy the uncompressed files to S3, then destroy the EC2 instance?

like image 398
Nyxynyx Avatar asked Mar 15 '12 12:03

Nyxynyx


People also ask

Can I upload zip file to S3 and unzip?

You can also upload the gzipped files to the same source bucket. Another way to do the same could be to first read the S3 file into the /tmp folder and then unzip it for further processing. However, this method might crash if you start getting multiple files in your S3 bucket at the same time.

How do I upload a zip file to Amazon S3?

To upload folders and files to an S3 bucketSign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that you want to upload your folders or files to. Choose Upload.


1 Answers

If you have an EC2 machine in the same region I would suggest you upload it there zipped and then it drop it to s3 from there unzipped. S3 cannot unzip it on its own as its all static.

Theres no charges between ec2 and s3 so ec2 can handle the unzipping and then write it out into your s3 bucket without additional transfer charges.

like image 136
Tarang Avatar answered Sep 21 '22 21:09

Tarang