Is it possible to copy all files from one S3 bucket to another with s3cmd?

Question

I'm pretty happy with s3cmd, but there is one issue: How to copy all files from one S3 bucket to another? Is it even possible?

EDIT: I've found a way to copy files between buckets using Python with boto:

from boto.s3.connection import S3Connection  def copyBucket(srcBucketName, dstBucketName, maxKeys = 100):   conn = S3Connection(awsAccessKey, awsSecretKey)    srcBucket = conn.get_bucket(srcBucketName);   dstBucket = conn.get_bucket(dstBucketName);    resultMarker = ''   while True:     keys = srcBucket.get_all_keys(max_keys = maxKeys, marker = resultMarker)      for k in keys:       print 'Copying ' + k.key + ' from ' + srcBucketName + ' to ' + dstBucketName        t0 = time.clock()       dstBucket.copy_key(k.key, srcBucketName, k.key)       print time.clock() - t0, ' seconds'      if len(keys) < maxKeys:       print 'Done'       break      resultMarker = keys[maxKeys - 1].key 

Syncing is almost as straight forward as copying. There are fields for ETag, size, and last-modified available for keys.

Maybe this helps others as well.

Answer 1

s3cmd sync s3://from/this/bucket/ s3://to/this/bucket/

For available options, please use: $s3cmd --help

Answer 2

AWS CLI seems to do the job perfectly, and has the bonus of being an officially supported tool.

aws s3 sync s3://mybucket s3://backup-mybucket 

http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html