Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Amazon S3: How to get a list of folders in the bucket?

The only thing I've found, it's this method GET Bucket.
But I can't understand how can I get only a list of folders in the current folder. What prefix and delimiter do I need to use? Is that possible at all?

like image 803
kkost Avatar asked Nov 05 '15 12:11

kkost


People also ask

How do I list a folder in a bucket?

To list all files, located in a folder of an S3 bucket, use the s3 ls command, passing in the entire path to the folder and setting the --recursive parameter.

How do you show objects and folders in a bucket?

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the left navigation pane, choose Buckets. In the Buckets list, choose the name of the bucket in which your folder is stored. In the Objects list, select the check box next to the name of the folder.

Do S3 buckets have directories?

Directories don't actually exist within S3 buckets. The entire file structure is actually just one flat single-level container of files. The illusion of directories are actually created based on naming the files names like dirA/dirB/file .

Can we download S3 bucket folder?

Unfortunately AWS S3 Console doesn't have an option to download all the content of an S3 bucket at the moment, but there are other options like AWS CLI to do so. For instance: aws s3 sync s3://all_my_stuff_bucket . This command will start downloading all the objects in all_my_stuff_bucket to the current directory.


2 Answers

Using prefix of the/path/to/read/ (note that there is no leading slash, but there is a trailing slash), and delimiter of /, you'll find all the folders within that folder inside <CommonPrefixes>.

CommonPrefixes

A response can contain CommonPrefixes only if you specify a delimiter. When you do, CommonPrefixes contains all (if there are any) keys between Prefix and the next occurrence of the string specified by delimiter. In effect, CommonPrefixes lists keys that act like subdirectories in the directory specified by Prefix. For example, if prefix is notes/ and delimiter is a slash (/), in notes/summer/july, the common prefix is notes/summer/. All of the keys rolled up in a common prefix count as a single return when calculating the number of returns. See MaxKeys.

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html

like image 98
Michael - sqlbot Avatar answered Oct 13 '22 00:10

Michael - sqlbot


What Anthony is missing here is that a folder doesn't necessarily have a key associated with it. If a file is created in S3, and given a key like "folder/name.ext", S3 will display a "folder" folder, but it doesn't have a key, meaning you're not getting it in your results.

The only way to catch these folders is to look at the keys themselves, and regex the key name for the "/" character. If I knew C# a little better, I'd write you a code sample, but for reference here's a python example I wrote on another question.

like image 21
Nathan Hazzard Avatar answered Oct 13 '22 00:10

Nathan Hazzard