Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS S3 CLI multipart upload

When trying to upload a file to an AWS S3 bucket using the aws-cli, does it automatically use multipart upload on bigger files? Because all it outputs is

Completed 3.0 GiB/9.4 GiB (18.4 MiB/s) with 1 file(s) remaining

which does not make me any wiser.

I checked several other information sources and this this page tells me it goes automatically, while the Amazon help tells me otherwise.

like image 757
PinkFluffyUnicorn Avatar asked Aug 03 '17 07:08

PinkFluffyUnicorn


1 Answers

They do mention in AWS docs that the CLI commands automatically perform a multipart upload for large objects.

Excerpt from documentation:

All high-level commands that involve uploading objects into an Amazon S3 bucket (aws s3 cp, aws s3 mv, and aws s3 sync) automatically perform a multipart upload when the object is large.

Failed uploads cannot be resumed when using these commands. If the multipart upload fails due to a timeout or is manually cancelled by pressing CTRL+C, the AWS CLI cleans up any files created and aborts the upload. This process can take several minutes.

If the process is interrupted by a kill command or system failure, the in-progress multipart upload remains in Amazon S3 and must be cleaned up manually in the AWS Management Console or with the s3api abort-multipart-upload command.

like image 151
hjpotter92 Avatar answered Sep 21 '22 07:09

hjpotter92