Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Enable Lambda function to an S3 bucket using cloudformation

We are creating an S3 bucket using a CloudFormation template. I would like to associate (Add an event to S3 bucket) a Lambda function whenever a file is added to the S3 bucket.

How is it possible through CloudFormation templates. What are the properties which needs to be used in CloudFormation.

like image 621
shiv455 Avatar asked Mar 31 '16 16:03

shiv455


People also ask

How do I give Lambda access to S3 bucket?

In order to grant a Lambda function access to an S3 Bucket, we have to attach an IAM policy to the function's execution role. The policy should grant permissions for all the Actions the function needs to perform on the specified bucket.

How do you trigger Lambda function in CloudFormation?

AWS CloudFormation invokes your Lambda function asynchronously with an event that includes a callback URL. The function is responsible for returning a response to the callback URL that indicates success or failure. For the full response syntax, see Custom resource response objects.


2 Answers

Here's a complete, self-contained CloudFormation template that demonstrates how to trigger a Lambda function whenever a file is added to an S3 bucket:

Launch Stack

Description: Upload an object to an S3 bucket, triggering a Lambda event, returning the object key as a Stack Output. Parameters:   Key:     Description: S3 Object key     Type: String     Default: test   Body:     Description: S3 Object body content     Type: String     Default: TEST CONTENT   BucketName:     Description: S3 Bucket name     Type: String Resources:   Bucket:     Type: AWS::S3::Bucket     DependsOn: BucketPermission     Properties:       BucketName: !Ref BucketName       NotificationConfiguration:         LambdaConfigurations:         - Event: 's3:ObjectCreated:*'           Function: !GetAtt BucketWatcher.Arn   BucketPermission:     Type: AWS::Lambda::Permission     Properties:       Action: 'lambda:InvokeFunction'       FunctionName: !Ref BucketWatcher       Principal: s3.amazonaws.com       SourceAccount: !Ref "AWS::AccountId"       SourceArn: !Sub "arn:aws:s3:::${BucketName}"   BucketWatcher:     Type: AWS::Lambda::Function     Properties:       Description: Sends a Wait Condition signal to Handle when invoked       Handler: index.handler       Role: !GetAtt LambdaExecutionRole.Arn       Code:         ZipFile: !Sub |           exports.handler = function(event, context) {             console.log("Request received:\n", JSON.stringify(event));             var responseBody = JSON.stringify({               "Status" : "SUCCESS",               "UniqueId" : "Key",               "Data" : event.Records[0].s3.object.key,               "Reason" : ""             });             var https = require("https");             var url = require("url");             var parsedUrl = url.parse('${Handle}');             var options = {                 hostname: parsedUrl.hostname,                 port: 443,                 path: parsedUrl.path,                 method: "PUT",                 headers: {                     "content-type": "",                     "content-length": responseBody.length                 }             };             var request = https.request(options, function(response) {                 console.log("Status code: " + response.statusCode);                 console.log("Status message: " + response.statusMessage);                 context.done();             });             request.on("error", function(error) {                 console.log("send(..) failed executing https.request(..): " + error);                 context.done();             });             request.write(responseBody);             request.end();           };       Timeout: 30       Runtime: nodejs4.3   Handle:     Type: AWS::CloudFormation::WaitConditionHandle   Wait:     Type: AWS::CloudFormation::WaitCondition     Properties:       Handle: !Ref Handle       Timeout: 300   S3Object:     Type: Custom::S3Object     Properties:       ServiceToken: !GetAtt S3ObjectFunction.Arn       Bucket: !Ref Bucket       Key: !Ref Key       Body: !Ref Body   S3ObjectFunction:     Type: AWS::Lambda::Function     Properties:       Description: S3 Object Custom Resource       Handler: index.handler       Role: !GetAtt LambdaExecutionRole.Arn       Code:         ZipFile: !Sub |           var response = require('cfn-response');           var AWS = require('aws-sdk');           var s3 = new AWS.S3();           exports.handler = function(event, context) {             console.log("Request received:\n", JSON.stringify(event));             var responseData = {};             if (event.RequestType == 'Create') {               var params = {                 Bucket: event.ResourceProperties.Bucket,                 Key: event.ResourceProperties.Key,                 Body: event.ResourceProperties.Body               };               s3.putObject(params).promise().then(function(data) {                 response.send(event, context, response.SUCCESS, responseData);               }).catch(function(err) {                 console.log(JSON.stringify(err));                 response.send(event, context, response.FAILED, responseData);               });             } else if (event.RequestType == 'Delete') {               var deleteParams = {                 Bucket: event.ResourceProperties.Bucket,                 Key: event.ResourceProperties.Key               };               s3.deleteObject(deleteParams).promise().then(function(data) {                 response.send(event, context, response.SUCCESS, responseData);               }).catch(function(err) {                 console.log(JSON.stringify(err));                 response.send(event, context, response.FAILED, responseData);               });             } else {               response.send(event, context, response.SUCCESS, responseData);             }           };       Timeout: 30       Runtime: nodejs4.3   LambdaExecutionRole:     Type: AWS::IAM::Role     Properties:       AssumeRolePolicyDocument:         Version: '2012-10-17'         Statement:         - Effect: Allow           Principal: {Service: [lambda.amazonaws.com]}           Action: ['sts:AssumeRole']       Path: /       ManagedPolicyArns:       - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"       Policies:       - PolicyName: S3Policy         PolicyDocument:           Version: '2012-10-17'           Statement:             - Effect: Allow               Action:                 - 's3:PutObject'                 - 'S3:DeleteObject'               Resource: !Sub "arn:aws:s3:::${BucketName}/${Key}" Outputs:   Result:     Value: !GetAtt Wait.Data 
like image 163
wjordan Avatar answered Nov 02 '22 19:11

wjordan


You need a NotificationConfiguration property in your CloudFormation template. Unfortunately, it seems to require the bucket to already exist. To get around this, you can create an initial stack, then update it with the NotificationConfiguration. For example:

    // template1.json     {       "AWSTemplateFormatVersion": "2010-09-09",       "Parameters": {         "mylambda": {           "Type": "String"         }       },       "Resources": {         "bucketperm": {           "Type": "AWS::Lambda::Permission",           "Properties" : {             "Action": "lambda:InvokeFunction",             "FunctionName": {"Ref": "mylambda"},             "Principal": "s3.amazonaws.com",             "SourceAccount": {"Ref": "AWS::AccountId"},             "SourceArn": { "Fn::Join": [":", [                 "arn", "aws", "s3", "" , "", {"Ref" : "mybucket"}]]             }           }         },         "mybucket": {           "Type": "AWS::S3::Bucket"         }       }     }      // template2.json -- adds the NotificationConfiguration     {       "AWSTemplateFormatVersion": "2010-09-09",       "Parameters": {         "mylambda": {           "Type": "String"         }       },       "Resources": {         "bucketperm": {           "Type": "AWS::Lambda::Permission",           "Properties" : {             "Action": "lambda:InvokeFunction",             "FunctionName": {"Ref": "mylambda"},             "Principal": "s3.amazonaws.com",             "SourceAccount": {"Ref": "AWS::AccountId"},             "SourceArn": { "Fn::Join": [":", [                 "arn", "aws", "s3", "" , "", {"Ref" : "mybucket"}]]             }           }         },         "mybucket": {           "Type": "AWS::S3::Bucket",           "Properties": {             "NotificationConfiguration": {               "LambdaConfigurations": [                 {                   "Event" : "s3:ObjectCreated:*",                   "Function" : {"Ref": "mylambda"}                 }               ]             }           }         }       }     } 

You can use the AWS CLI tool to create the stack like this:

    $ aws cloudformation create-stack --stack-name mystack --template-body file://template1.json --parameters ParameterKey=mylambda,ParameterValue=<lambda arn>     # wait until stack is created     $ aws cloudformation update-stack --stack-name mystack --template-body file://template2.json --parameters ParameterKey=mylambda,ParameterValue=<lambda arn> 
like image 45
ataylor Avatar answered Nov 02 '22 18:11

ataylor