Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I do an S3 copy between regions using aws cli?

It was far to difficult to figure this out. It wasn't obvious to me and lots of explanations left out key details. I will answer this with the solution. Sorry if it seems obvious to you, but given how many searches and experiments it took me to do this, I think it is quite worthwhile to show others how to do it.

like image 251
Solx Avatar asked Jan 28 '16 15:01

Solx


People also ask

Is S3 replicated across regions?

Amazon S3 now supports cross-region replication, a new feature that automatically replicates data across AWS regions. With cross-region replication, every object uploaded to an S3 bucket is automatically replicated to a destination bucket in a different AWS region that you choose.

How can data be replicated in different AWS regions in S3?

With Amazon S3 Replication, you can configure Amazon S3 to automatically replicate S3 objects across different AWS Regions by using S3 Cross-Region Replication (CRR) or between buckets in the same AWS Region by using S3 Same-Region Replication (SRR).

How can I copy S3 objects from another AWS account?

Create and attach an S3 bucket policy. Sign in to the AWS Management Console for your source account and open the Amazon S3 console. Choose your source S3 bucket and then choose Permissions. Under Bucket policy, choose Edit and then paste the bucket policy from the sourcebucket-policy.


1 Answers

UPDATE: According to a commenter, the extra parameters I show here may no longer be needed. I am not currently working with AWS, so I don't have a way to verify it. Anyway, I didn't change the rest of this post in case it is still needed in some case.

The trick is being explicit about both the source and destination regions. They might not always be required, but it doesn't hurt to always show them:

$ aws s3 cp s3://my-source-bucket-in-us-west-2/ \
      s3://my-target-bucket-in-us-east-1/ \
      --recursive --source-region us-west-2 --region us-east-1

Or on Windows

> aws s3 cp s3://my-source-bucket-in-us-west-2/ ^
          s3://my-target-bucket-in-us-east-1/ ^
          --recursive --source-region us-west-2 --region us-east-1
like image 160
Solx Avatar answered Oct 04 '22 16:10

Solx