Syncing of S3 to local directory in Python

Question

I am trying to sync S3 bucket:

 s3://xxxxxxxx-data/ds_2/accounts/xxxxxx

to my local:

/abc/def

in Python 3. Could someone please suggest me how to do it?

I am not able to figure out this on the basis of current questions available. Thanks in advance

Answer 1

You could call the s3 sync command from python with os.system:

import os

cmd = 'aws s3 sync s3://source-bucket/ my-dir'
os.system(cmd)

Answer 2

There is no s3 sync feature in boto3 as there is in AWS CLI. There are open issues for that though:

• Should support S3 Bucket Sync

• Directory upload/download with boto3

So hopeful, one day they will be implemented.

In the issues you can find code snippets of functions others made to get similar functionality. There are also plenty of similar code in other places.