Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

create folder inside S3 bucket using Cloudformation

I'm able to create an S3 bucket using cloudformation but would like to create a folder inside an S3 bucket..like

<mybucket>--><myfolder> 

Please let me know the template to be used to create a folder inside a bucket ...both should be created at the sametime...

I'm Using AWS lambda as below

stackname = 'myStack' client = boto3.client('cloudformation') response = client.create_stack(     StackName= (stackname),     TemplateURL= 'https://s3.amazonaws.com/<myS3bucket>/<myfolder>/nestedstack.json',     Parameters=<params> ) 
like image 413
shiv455 Avatar asked Apr 28 '16 14:04

shiv455


People also ask

Can we create folder in S3 bucket using CloudFormation?

You can use the CloudFormation template in the following resolution to use custom resources with an S3 bucket. Consider the following: The template allows you to create folders in S3 buckets. Amazon S3 has a flat structure, but supports the folder concept as a means of grouping objects.

Can we create folder inside S3 bucket?

You can have folders within folders, but not buckets within buckets. You can upload and copy objects directly into a folder. Folders can be created, deleted, and made public, but they cannot be renamed.

Does AWS S3 CP create folder?

The Amazon S3 implements folder object creation by creating a zero-byte object. If you see a file in the console you will see the key of the file also has the folder reference in the key – test-folder/hdfs-0.0.

Does CloudFormation create S3 bucket?

If you specify a template file stored locally, CloudFormation uploads it to an S3 bucket in your AWS account. CloudFormation creates a bucket for each region in which you upload a template file. The buckets are accessible to anyone with Amazon Simple Storage Service (Amazon S3) permissions in your AWS account.


Video Answer


1 Answers

AWS doesn't provide an official CloudFormation resource to create objects within an S3 bucket. However, you can create a Lambda-backed Custom Resource to perform this function using the AWS SDK, and in fact the gilt/cloudformation-helpers GitHub repository provides an off-the-shelf custom resource that does just this.

As with any Custom Resource setup is a bit verbose, since you need to first deploy the Lambda function and IAM permissions, then reference it as a custom resource in your stack template.

First, add the Lambda::Function and associated IAM::Role resources to your stack template:

"S3PutObjectFunctionRole": {   "Type": "AWS::IAM::Role",   "Properties": {     "AssumeRolePolicyDocument": {       "Version" : "2012-10-17",       "Statement": [         {           "Effect": "Allow",           "Principal": {             "Service": [ "lambda.amazonaws.com" ]           },           "Action": [ "sts:AssumeRole" ]         }       ]     },     "ManagedPolicyArns": [       { "Ref": "RoleBasePolicy" }     ],     "Policies": [       {         "PolicyName": "S3Writer",         "PolicyDocument": {           "Version" : "2012-10-17",           "Statement": [             {               "Effect": "Allow",               "Action": [                 "s3:DeleteObject",                 "s3:ListBucket",                 "s3:PutObject"               ],               "Resource": "*"             }           ]         }       }     ]   } }, "S3PutObjectFunction": {   "Type": "AWS::Lambda::Function",   "Properties": {     "Code": {       "S3Bucket": "com.gilt.public.backoffice",       "S3Key": "lambda_functions/cloudformation-helpers.zip"     },     "Description": "Used to put objects into S3.",     "Handler": "aws/s3.putObject",     "Role": {"Fn::GetAtt" : [ "S3PutObjectFunctionRole", "Arn" ] },     "Runtime": "nodejs",     "Timeout": 30   },   "DependsOn": [     "S3PutObjectFunctionRole"   ] }, 

Then you can use the Lambda function as a Custom Resource to create your S3 object:

"MyFolder": {   "Type": "Custom::S3PutObject",   "Properties": {     "ServiceToken": { "Fn::GetAtt" : ["S3PutObjectFunction", "Arn"] },     "Bucket": "mybucket",     "Key": "myfolder/"   } }, 

You can also use the same Custom Resource to write a string-based S3 object by adding a Body parameter in addition to Bucket and Key (see the docs).

like image 197
wjordan Avatar answered Sep 21 '22 21:09

wjordan