Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to copy S3 objects between regions with Amazon AWS PHP SDK?

I'm trying to copy Amazon AWS S3 objects between two buckets in two different regions with Amazon AWS PHP SDK v3. This would be a one-time process, so I don't need cross-region replication. Tried to use copyObject() but there is no way to specify the region.

$s3->copyObject(array(
    'Bucket'     => $targetBucket,
    'Key'        => $targetKeyname,
    'CopySource' => "{$sourceBucket}/{$sourceKeyname}",
));

Source: http://docs.aws.amazon.com/AmazonS3/latest/dev/CopyingObjectUsingPHP.html

like image 924
bart Avatar asked Feb 23 '17 11:02

bart


2 Answers

You don't need to specify regions for that operation. It'll find out the target bucket's region and copy it.

But you may be right, because on AWS CLI there is source region and target region attributes which do not exist on PHP SDK. So you can accomplish the task like this:

  1. Create an interim bucket in the source region.
  2. Create the bucket in the target region.
  3. Configure replication from the interim bucket to target one.
  4. On interim bucket set expiration rule, so files will be deleted after a short time automatically from the interim bucket.
  5. Copy objects from source bucket to interim bucket using PHP SDK.
  6. All your objects will also be copied to another region.
  7. You can remove the interim bucket one day later.

Or use just cli and use this single command:

aws s3 cp s3://my-source-bucket-in-us-west-2/ s3://my-target-bucket-in-us-east-1/ --recursive --source-region us-west-2 --region us-east-1
like image 61
Çağatay Gürtürk Avatar answered Oct 05 '22 01:10

Çağatay Gürtürk


Different region bucket could also be different account. What others had been doing was to copy off from one bucket and save the data temporary locally, then upload to different bucket with different credentials. (if you have two regional buckets with different credentials).

Newest update from CLI tool allows you to copy from bucket to bucket if it's under the same account. Using something like what Çağatay Gürtürk mentioned.

like image 40
Chris Huang Avatar answered Oct 05 '22 02:10

Chris Huang