Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Create directories in Amazon S3 using python, boto3

Tags:

I know S3 buckets not really have directories because the storage is flat. But it is possible to create directories programmaticaly with python/boto3, but I don't know how. I saw this on a documentary :

"Although S3 storage is flat: buckets contain keys, S3 lets you impose a directory tree structure on your bucket by using a delimiter in your keys. For example, if you name a key ‘a/b/f’, and use ‘/’ as the delimiter, then S3 will consider that ‘a’ is a directory, ‘b’ is a sub-directory of ‘a’, and ‘f’ is a file in ‘b’."

I can create just files in the a S3 Bucket by :

    self.client.put_object(Bucket=bucketname,Key=filename) 

but I don't know how to create a directory.

like image 537
Steve Ritz Avatar asked Dec 10 '15 00:12

Steve Ritz


People also ask

Can we create folder in S3 bucket using Python?

1 Answer. Yes, there is. It is just 5 lines of code where one line is importing boto3. In folder_name, provide all the folder names you want to add.

What is the difference between Boto3 client and resource?

00:00 Boto3's primary function is to make AWS API calls for you. It extracts these APIs in two main ways: clients and resources. Clients give you low-level service access, while resources provide an object-oriented way of working with these services.

Does AWS S3 have folders?

In Amazon S3, folders are used to group objects and organize files. Unlike a traditional file system, Amazon S3 doesn't use hierarchy to organize its objects and files. Amazon S3 console supports the folder concept only as a means of grouping (and displaying) objects.


2 Answers

Just a little modification in key name is required. self.client.put_object(Bucket=bucketname,Key=filename)

this should be changed to

self.client.put_object(Bucket=bucketname,Key=directoryname/filename)

Thats all.

like image 71
Hardeep Singh Avatar answered Sep 29 '22 09:09

Hardeep Singh


If you read the API documentation You should be able to do this.

import boto3  s3 = boto3.client("s3")  BucketName = "mybucket" myfilename = "myfile.dat" KeyFileName = "/a/b/c/d/{fname}".format(fname=myfilename)  with open(myfilename) as f :    object_data = f.read()   client.put_object(Body=object_data, Bucket=BucketName, Key=KeyFileName) 

Honestly, it is not a "real directory", but preformat string structure for organisation.

like image 43
mootmoot Avatar answered Sep 29 '22 11:09

mootmoot