C# - AWS S3 uploading a file using PutObject or TransferUtility.Upload method. Which is the better way to go with?

Question

I know there are two methods available to upload files in AWS S3 (i.e. PutObject and TransferUtility.Upload). Can someone please explain which one to use?

FYI, I have files ranging from 1kb to 250MB.

Thanks in advance.

Answer 1

Amazon deprecated the S3 Transfer Manager and migrated to the new Transfer Utility.The Transfer Utility is a simple interface for handling the most common uses of S3.It has a single constructor, which requires an instance of AmazonS3Client. Working with it is so easy and let the develpers perform all operations with less code.

Following are key features of using Transfer Utility over Transfer Manager

• When uploading large files, TransferUtility uses multiple threads to upload multiple parts of a single upload at once.When dealing with large content sizes and high bandwidth, this can increase throughput significantly.TransferUtility detects if a file is large and switches into multipart upload mode. The multipart upload gives the benefit of better performance as the parts can be uploaded simultaneously as well, and if there is an error, only the individual part has to be retried.
• Mostly we people try to upload large files on S3 that take too much time to upload,at those situations we required progress information such as the total number of bytes transferred and remaining amount of data to transfer.To track current progress of transfer with the Transfer Manager, developers pass an S3ProgressListener callback to upload or download, which periodically fires the method below.
• Pausing transfers using the Transfer Manager is not possible with stream based uploads or downloads.But Transfer Utility provide us pause and resume option, it also has one single-file-based method for uploads, and downloads.

transferUtility.upload(MY_BUCKET,OBJECT_KEY,FILE_TO_UPLOAD) transferUtility.download(MY_BUCKET,OBJECT_KEY,FILE_TO_UPLOAD)

• The Transfer Manager only requires the INTERNET permission. However, since the Transfer Utility automatically detects network state and pauses/resumes transfers based on the network state pause functionality to the Transfer Utility is easy, since all transfers can be paused and resumed.A transfer is paused because of a loss of network connectivity, it will automatically be resumed and there is no action you need to take.Transfers that are automatically paused and waiting for network connectivity will have the state.Additionally, the Transfer Utility stores all of the metadata about transfers to the local SQLite database, so developers do not need to persist anything.

Note : Every thing else is good.But Transfer Utility does not support a copy() API.To accomplish it use AmazonS3Client class copyObject() method.

Answer 2

Based in Amazon docs, I would stick with TransferUtility.Upload:

Provides a high level utility for managing transfers to and from Amazon S3.

TransferUtility provides a simple API for uploading content to and downloading content from Amazon S3. It makes extensive use of Amazon S3 multipart uploads to achieve enhanced throughput, performance, and reliability.

When uploading large files by specifying file paths instead of a stream, TransferUtility uses multiple threads to upload multiple parts of a single upload at once. When dealing with large content sizes and high bandwidth, this can increase throughput significantly.

But please be aware of possible concurrency issues and the recommendation about using BeginUpload (the asynchronous version), like in this related post