Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to copy the object from s3 to s3 using node.js

I would like to know how to copy the object from s3 to s3 using node.js With the aws s3 command, It could be executed as follows.

s3 cp --recursive s3://xx/yy  s3://zz/aa

http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#copyObject-property

I refer to above link,But I don't know how to do it using node.js

any Idea?

like image 205
negabaro Avatar asked Oct 26 '17 06:10

negabaro


People also ask

Why can't I copy an object between two Amazon S3 buckets?

To copy an object between buckets, you must make sure that the correct permissions are configured. To copy an object between buckets in the same AWS account, you can set permissions using IAM policies.

Can I copy S3 bucket to another account?

You then create an IAM policy in your destination account that allows a user to perform PutObject and GetObject actions on the source S3 bucket. Finally, you run copy and sync commands to transfer data from the source S3 bucket to the destination S3 bucket. Accounts own the objects that they upload to S3 buckets.


2 Answers

If you just want to copy one object copyObject API.

 var params = {
  Bucket: "destinationbucket", 
  CopySource: "/sourcebucket/sourceKeyName", 
  Key: "targetKeyName"
 };
 s3.copyObject(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else     console.log(data);           // successful response
 });

If you want to perform recursively for all objects in a bucket, then

  1. List all the object keys in a bucket using the listObjectsV2 API.

  2. If versioning is enabled in the source-bucket and you want to copy a specific version of the key, Invoke the listObjectVersions API as well and get the Version-Id for each S3 Key.

    NOTE: If versioning is not enabled, then you can ignore STEP-2.

  3. Call copyObject for each S3 Key and the Version-Id obtained in Step-1 and Step-2 respectively. Version-id is optional.

like image 72
Madhukar Mohanraju Avatar answered Oct 04 '22 14:10

Madhukar Mohanraju


If you want to really move (so not just copy, but in addition remove the source file)

const moveAndDeleteFile = async (file,inputfolder,targetfolder) => {
    const s3 = new AWS.S3();

    const copyparams = {
        Bucket : bucketname,
        CopySource : bucketname + "/" + inputfolder + "/" + file, 
        Key : targetfolder + "/" + file
    };

    await s3.copyObject(copyparams).promise();

    const deleteparams = {
        Bucket : bucketname,
        Key : inputfolder + "/" + file
    };

    await s3.deleteObject(deleteparams).promise();
    ....
}
like image 31
Tobi Avatar answered Oct 04 '22 14:10

Tobi