Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Transferring 100,000 images to S3. What's the quickest way? [closed]

Tags:

amazon-s3

I'm looking for the fastest way to transfer ~100,000 images from a server to my Amazon S3 bucket.

Would that be to loop through every image and do a separate PUT request? I was hoping there was a quicker way, but haven't found anything.

like image 875
makeee Avatar asked Jun 18 '09 01:06

makeee


2 Answers

Maybe try the snail mail?

Amazon Import/Export

like image 189
Hao Wooi Lim Avatar answered Sep 25 '22 13:09

Hao Wooi Lim


s3funnel. I used this last week to transfer around 3,000,000 images to S3. It is multi-threaded, so tends to be very fast. I tested a number of commercial and open-source tools, and s3funnel is by far the best IMO.

like image 42
Mike Ryan Avatar answered Sep 25 '22 13:09

Mike Ryan