Menu
I am in the process of converting a website to upload user submitted photos and videos directly to Amazon S3 instead of to the local server.
When I worked with small image files, it seemed fine... but now that I try with larger images say over 1MB, the upload speed is too slow and unusable.
What are the common strategies to overcome this? Do websites like YouTube first upload to their local server and then move to Google Cloud or something? If I have thousands of users uploading at the same time, my local server won't be able to store so many files hence I was moving to S3 before realising how slow it was.
Any suggestions please?
907
Consider the following options for improving the performance of uploads and optimizing multipart uploads: If you're using the AWS CLI, customize the upload configurations. Enable Amazon S3 Transfer Acceleration.
You can increase your read or write performance by using parallelization. For example, if you create 10 prefixes in an Amazon S3 bucket to parallelize reads, you could scale your read performance to 55,000 read requests per second. Similarly, you can scale write operations by writing to multiple prefixes.
Upload speed to AWS S3 tops out at 2.3 Mbps.
Large object size For very large Amazon S3 objects, you might notice slow download times as your web browser tries to download the entire object. Instead, try downloading large objects with a ranged GET request using the Amazon S3 API.
I would take the following steps:
Enable Transfer Acceleration on your S3 bucket.
Change your application to upload files in multiple parts, using S3 Multipart Upload, and use multi-threading to upload more than one part at a time.
173
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
