Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is TransferManager reusable after upload?

Tags:

aws-java-sdk

I'm using TransferManager to upload a file to my s3 bucket. While the documentation says to reuse one instance wherever possible, it also says that shutdown should be called once the transfer is complete.

Relevant portion of the docs:

TransferManager is responsible for managing resources such as connections and threads; share a single instance of TransferManager whenever possible. TransferManager, like all the client classes in the AWS SDK for Java, is thread safe. Call TransferManager.shutdownNow() to release the resources once the transfer is complete.

Is this to be taken literally, or can we reuse the object and call shutdown when the application is closed?

A bit of background: We were earlier using the same s3 client to create one TransactionManager per upload, but we started getting AbortedException thrown on some of the uploads.

like image 765
Aditya Vivek Avatar asked Mar 06 '18 07:03

Aditya Vivek


1 Answers

Answering my own question here: We've reused the transfer manager in testing and production without any issues.

AWS SDK related queries are answered on this chat: https://gitter.im/aws/aws-sdk-java

Searching chat logs with "aws client reusable" should point to the relevant comment by someone on the team.

Transcript of the relevant comments in the chat linked above:

soutchay Feb 13 2018 17:08 - Hi everyone, we have a Spring Boot Java application using aws sdk to upload objects to S3. We have a TransferManager bean and are using it to perform an upload and a waitForUploadResult. The documentation on the TransferManager says to call shutdownNow()after waitForUploadResult completes.

AdityaVivek Mar 06 2018 11:08 - Hi everyone, I have the same doubt as @soutchay , can TransferManager be reused after upload or does it have to be shutdown immediately?

jaramd Mar 06 2018 04:17 - Hi Aditya, you can reuse the TransferManager object, which is thread safe actually. It has a executor pool with control number of threads. So you cannot span more that thread pool size.

like image 93
Aditya Vivek Avatar answered Oct 10 '22 06:10

Aditya Vivek