I have a static site that I am deploying to s3
and then using CloudFront
to distribute to users. After I build the site, I want to push the new build to s3
. I found 2 approaches to do that.
aws s3 cp --recursive ./public/ s3://bucket-name --cache-control 'public, max-age=300, s-maxage=31536000'
aws s3 sync --delete ./public/ s3://bucket-name --cache-control 'public, max-age=300, s-maxage=31536000'
I am planning to deploy once or twice every week.
I want to know which of these is less expensive (money)? To be more clear, I want to know which among these will cost me less in the long run?
I tried reading the docs, but I was not able to figure out the differences. Please help me with this.
S3 cp – Will read all the files from the source location and write into the new location. S3 sync – Will scan the new location and only overwrite the file from the source location if the file is newly created or updated(via file size and modified timestamp comparison).
The s3 sync command synchronizes the contents of a bucket and a directory, or the contents of two buckets. Typically, s3 sync copies missing or outdated files or objects between the source and target.
The cp command initiates a copy operation to or from Amazon S3. The --recursive option instructs the AWS CLI for Amazon S3 to descend into subdirectories on the source. The --quiet option instructs the AWS CLI for Amazon S3 to print only errors rather than a line for each file copied.
To reduce latency, reduce the geographical distance between the instance and your Amazon S3 bucket. If the instance is in the same Region as the source bucket, then set up an Amazon Virtual Private Cloud (Amazon VPC) endpoint for S3. VPC endpoints can help improve overall performance.
One thing to note is that aws s3 cp --recursive
and aws s3 sync --delete
have different behaviors.
aws s3 cp
will copy all files, even if they already exist in the destination area. It also will not delete files from your destination if they are deleted from the source.
aws s3 sync
looks at the destination before copying files over and only copies over files that are new and updated. The --delete
flag also will delete things at the destination if they were removed in source.
The sync
command is what you want as it is designed to handle keeping two folders in sync while copying the minimum amount of data. Sync should result in less data being pushed into S3 bucket so that should have a less cost overall.
To give a counterexample, a use case where aws s3 cp
outperforms and is lower cost than sync
is if you just need to transfer files and you know all the files are new to the destination. This is more performant and lower cost because the code is not checking the destination if things exist before starting the transfer.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With