Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to copy s3 object from one bucket to another using python boto3

I want to copy a file from one s3 bucket to another. I get the following error:

s3.meta.client.copy(source,dest)
TypeError: copy() takes at least 4 arguments (3 given)

I'am unable to find a solution by reading the docs. Here is my code:

#!/usr/bin/env python import boto3 s3 = boto3.resource('s3') source= { 'Bucket' : 'bucketname1','Key':'objectname'} dest ={ 'Bucket' : 'Bucketname2','Key':'backupfile'} s3.meta.client.copy(source,dest) 
like image 621
vishal Avatar asked Nov 24 '17 07:11

vishal


People also ask

How do I copy files from one S3 bucket to another S3 bucket in Python?

import boto3 s3 = boto3. resource('s3') copy_source = { 'Bucket': 'mybucket', 'Key': 'mykey' } bucket = s3. Bucket('otherbucket') bucket. copy(copy_source, 'otherkey') # This is a managed transfer that will perform a multipart copy in # multiple threads if necessary.

How do I copy a file from one bucket to another in Python?

You can use the Boto3 Session and bucket. copy() method to copy files between S3 buckets. You need your AWS account credentials for performing copy or move operations.

Why can't I copy an object between two Amazon S3 buckets?

If the object that you can't copy between buckets is owned by another account, then the object owner can do one of the following: The object owner can grant the bucket owner full control of the object. After the bucket owner owns the object, the bucket policy applies to the object.


2 Answers

You can try:

import boto3 s3 = boto3.resource('s3') copy_source = {       'Bucket': 'mybucket',       'Key': 'mykey'     } bucket = s3.Bucket('otherbucket') bucket.copy(copy_source, 'otherkey') 

or

import boto3 s3 = boto3.resource('s3') copy_source = {     'Bucket': 'mybucket',     'Key': 'mykey'  } s3.meta.client.copy(copy_source, 'otherbucket', 'otherkey') 

Note the difference in the parameters

like image 55
Adarsh Avatar answered Sep 23 '22 18:09

Adarsh


Since you are using s3 service resource, why not use its own copy method all the way?

#!/usr/bin/env python import boto3 s3 = boto3.resource('s3') source= { 'Bucket' : 'bucketname1', 'Key': 'objectname'} dest = s3.Bucket('Bucketname2') dest.copy(source, 'backupfile') 
like image 34
hjpotter92 Avatar answered Sep 20 '22 18:09

hjpotter92