I have a large size exe file (> 100 MB) which I need to upload to S3. Now since I have a reliable network, I want to make this process faster by dividing the file into chunks and then uploading these chunks in parallel. But I don't know how to merge these chunks back together to get the original exe?
Although python provides a way to upload a file into chunks but those chunks are uploaded serially. Is there any way to upload these chunks in parallel and then merge them back together?
S3 already has support for this.
It's called a multipart upload: http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html
Use a client (for example s3cmd http://s3tools.org/kb/item13.htm) that knows how to do this or use do it yourself through the SDK and your favorite language.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With