How to programmatically give s3 bucket permission to invoke lambda?

Question

What I am doing

Currently I am trying to dynamically create an event on a bucket to call a lambda function, whenever a trigger like s3:ObjectCreated:* or s3:ObjectRemoved:* occurs.

My current code

I have the following code

import boto3

client = boto3.client('s3',
                      aws_access_key_id="...",
                      aws_secret_access_key="...",
                      region_name="us-east-1")


response = client.put_bucket_notification(
    Bucket='my_test_bucket',
    NotificationConfiguration={
    'CloudFunctionConfiguration': {
            'Id': event_name,
            'Events': [
                's3:ObjectCreated:*',
                's3:ObjectRemoved:*',
            ],
            'CloudFunction': 'arn:aws:lambda:...',

        }
    }
)

My problem

However, when I try to run the code, I get the following error

An error occurred (InvalidArgument) when calling the PutBucketNotification 
operation: Unable to validate the following destination configurations

Not authorized to invoke function [arn:aws:lambda:...]

My question

How do I programmatically give my_test_bucket permission to run arn:aws:lambda:...?

Answer 1

Please refer to the official documentation, the method you are using is deprecated and while it is maintained for backwards compatibility, it may not function as expected. See here for more information; you should be using PutBucketNotificationConfiguration which is bucket_notification.put().

Nonetheless, the error that you are seeing is:

Not authorized to invoke function [arn:aws:lambda:...]

This is because when your Lambda executes, the service invoking it needs permission to invoke it; you simply need to give S3 permission to invoke your Lambda. You do this by creating a Resource Policy for your Lambda.

In Python you're going to do this something like this:

 client = boto3.client('lambda')
 response = client.add_permission(
     FunctionName='arn:aws:lambda:...',
     StatementId='LambdaInvoke',
     Action='lambda:InvokeFunction',
     Principal='s3.amazonaws.com',
     SourceArn='arn:aws:s3:::my_test_bucket',
     SourceAccount='123456789012'
 )

Don't forget to replace the function-name with your functions ARN, as well as the source-arn to be your S3 buckets ARN, and source-account to be the source account number.

An alternative way, if you wanted to do it from the CLI, you can do it like this:

aws lambda add-permission --function-name arn:aws:lambda:... --principal s3.amazonaws.com --statement-id S3EventTrigger --action "lambda:InvokeFunction" --source-arn arn:aws:s3:::my_test_bucket --source-account 123456789012

Both of these methods will attach a resource policy, that looks like the policy below, to your Lambda, allowing your S3 bucket to invoke it.

{"Policy" : "{"Version":"2012-10-17","Id":"default","Statement":[{"Sid":"S3EventTrigger","Effect":"Allow","Principal":{"Service":"s3.amazonaws.com"},"Action":"lambda:InvokeFunction","Resource":"arn:aws:lambda:...","Condition":{"StringEquals":{"AWS:SourceAccount":"123456789012"},"ArnLike":{"AWS:SourceArn":"arn:aws:s3:::my_test_bucket"}}}]}" ,
"RevisionId" : "999a9999-99ab-9999-a9a9-9999999a999a"}

Answer 2

I recommand you to read the official document https://docs.aws.amazon.com/lambda/latest/dg/access-control-resource-based.html

As it said, "When you use an AWS service to invoke your function, you grant permission in a statement on a resource-based policy."

Also, here is an official tutorial of lambda with S3 https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html.

As it said, you can grant permission by running command in cli:

$ aws lambda add-permission --function-name your-function-name --principal s3.amazonaws.com \
--statement-id some-unique-id --action "lambda:InvokeFunction" \
--source-arn arn:aws:s3:::sourcebucket \
--source-account bucket-owner-account-id