Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the Difference between file_upload() and put_object() when uploading files to S3 using boto3

I'm using boto3 and trying to upload files. It will be helpful if anyone will explain exact difference between file_upload() and put_object() s3 bucket methods in boto3 ?

  • Is there any performance difference?
  • Does anyone among these handles multipart upload feature in behind the scenes?
  • What are the best use cases for both?
like image 916
Tushar Niras Avatar asked May 02 '17 13:05

Tushar Niras


People also ask

How do you append data to an existing csv file in AWS S3 using Python boto3?

s3 has no append functionality. You need to read the file from s3, append the data in your code, then upload the complete file to the same key in s3.

What is the best way for the application to upload the large files in S3?

When you upload large files to Amazon S3, it's a best practice to leverage multipart uploads. If you're using the AWS Command Line Interface (AWS CLI), then all high-level aws s3 commands automatically perform a multipart upload when the object is large. These high-level commands include aws s3 cp and aws s3 sync.


1 Answers

The upload_file method is handled by the S3 Transfer Manager, this means that it will automatically handle multipart uploads behind the scenes for you, if necessary.

The put_object method maps directly to the low-level S3 API request. It does not handle multipart uploads for you. It will attempt to send the entire body in one request.

like image 59
garnaat Avatar answered Sep 25 '22 01:09

garnaat