create empty folder S3 ruby SDK

Question

I want to create an empty folder in amazon S3 using the ruby sdk. Ive read that there is no folder concept in S3, so theoretically to create a folder you would just create an empty object with a trailing "/"

s3 = Aws::S3::Client.new( region: 'eu-west-1',
                          credentials: creds)

s3.put_object(bucket: "my_bucket",
              key: "my_folder/")

Doing that creates an empty object on my bucket, but then if I try to upload a file like this:

s3.put_object(bucket: "my_bucket",
              key: "my_folder/myfile")

It doesnt create a file in my_folder. It maintains the old empty object, and creates a folder and a file. So after the two commands the bucket structure is:

my_bucket/
  my_folder
  my_folder/
    my_file 

Why is this happening? why does it create the object my_folder twice? How should I create an empty folder for later use?

Answer 1

The Amazon S3 virtual folders spring into existence when any key contains a '/'. When browsing a bucket in the S3 console, it scans object keys for common prefixes and then uses that prefix to show a subset of a bucket.

Given the following object keys:

• photos/family/reunion.jpg
• photos/family/vacation.jpg
• videos/funny.mp4

Then Amazon S3 would show the top level folders "photos" and "videos". If you delete the "videos/funny.mp4" object, then the "videos" directory would disappear.

Answer 2

As you say, S3 does not have the concept of a folder, "empty" or not, it just has objects which can have a "/" in them to mimic the naming convention of folders in a file system.

"my_folder" is not being treated twice, there are just two objects with a name that begins "my_folder/".

What you'll have to do if you want to persist with this paradigm of having-folders-where-there-are-none is to delete your "my_folder/" object as soon as you create the first "my_folder/*" object.

And you should really not be asking, "How should I create an empty folder for later use?" when you acknowledge that folders do not exist.

Answer 3

S3 behaves as expected. It does not support folders. When you create file with name my_folder/myfile, S3 creates object with key my_folder/myfile. The idea of "folders" is built around access with prefix. So that, you can list object in "folder" like that:

bucket.objects.with_prefix('my_folder').collect(&:key)
# => ["my_folder/myfile"]