Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can you list all folders in an S3 bucket?

I have a bucket containing a number of folders each folders contains a number of images. Is it possible to list all the folders without iterating through all keys (folders and images) in the bucket. I'm using Python and boto.

like image 910
nickponline Avatar asked May 29 '14 03:05

nickponline


People also ask

Do S3 Buckets have directories?

Directories don't actually exist within S3 buckets. The entire file structure is actually just one flat single-level container of files. The illusion of directories are actually created based on naming the files names like dirA/dirB/file .

How do I view contents of a S3 bucket?

To open the overview pane for an objectSign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that contains the object. In the Objects list, choose the name of the object for which you want an overview.

How can you download an S3 bucket including all folders and files?

To download an entire bucket to your local file system, use the AWS CLI sync command, passing it the s3 bucket as a source and a directory on your file system as a destination, e.g. aws s3 sync s3://YOUR_BUCKET . . The sync command recursively copies the contents of the source to the destination.


1 Answers

You can use list() with an empty prefix (first parameter) and a folder delimiter (second parameter) to achieve what you're asking for:

s3conn = boto.connect_s3(access_key, secret_key, security_token=token)
bucket = s3conn.get_bucket(bucket_name)
folders = bucket.list('', '/')
for folder in folders:
    print folder.name

Remark:
In S3 there is no such thing as "folders". All you have is buckets and objects.

The objects represent files. When you name a file: name-of-folder/name-of-file it will look as if it's a file: name-of-file that resides inside folder: name-of-folder - but in reality there's no such thing as the "folder".

You can also use AWS CLI (Command Line Interface):
the command s3ls <bucket-name> will list only the "folders" in the first-level of the bucket.

like image 128
Nir Alfasi Avatar answered Nov 01 '22 23:11

Nir Alfasi