AWS Lambda Policy Length Exceeded - adding rules to a lambda function

Question

I have a website that dynamically creates rules that have cron events attached. All of these rules are associated with and call a single lambda function.

I am using python and boto3 to generate the rules and apply them to the Lambda Function. (If seeing my python code that generates the rules and events would help, I'd be happy to include it here.)

---

This all works, however after using my website and creating about 68 rules I got this error:

PolicyLengthExceededException: An error occurred (PolicyLengthExceededException) when calling the AddPermission operation: The final policy size (20642) is bigger than the limit (20480).

Every-time I create a rule and its event, a permission needs to be added to the lambda's Function Policy, and after about 68 rules, the Function Policy gets too large.

How can I fix this?

---

Here is an example permission:

{
  "Sid": "<some_random_id_for_this_permission",
  "Effect": "Allow",
  "Principal": {
    "Service": "events.amazonaws.com"
  },
  "Action": "lambda:InvokeFunction",
  "Resource": "arn:aws:lambda:<some_arn_id>:function:_MyFunction",
  "Condition": {
    "ArnLike": {
      "AWS:SourceArn": "arn:aws:events<some_arn_id>:rule/my_rule_1"
    }
  }

The AWS:SourceArn is the unique value of the rule, so I've figured I could use * to give permission to all rules. So I tried putting as the value for the AWS:SourceArn:

arn:aws:events<some_arn_data>:rule/*

But on the function dashboard, down where the list of CloudWatch Events is at, all it said was:

The rule * could not be found.

---

Is there a way to make a permission that applies to all rules?

If not, is there a different way to solve this?

And if there is no way to directly solve this issue, I could create a separate lambda for each record on the website rather than create separate rules for each record that all point to one lambda. Is there any reason why creating a separate lambda would this would be a bad idea? Such as is there a limit to the amount of Lambda functions you can have?

Answer 1

The Lambda console creates what I believe to be an illusion in many cases that Lambda is aware of event sources that are "wired up" to trigger functions. In some cases, like DynamoDB streams and SQS, the Lambda service is aware of these things, because they trigger functions because part of the Lambda service is actually polling those services in the background in order to "consume" them -- while other services are pushing their events to Lambda as they occur (e.g. S3, CloudWatch events).

Here's my justification for that conclusion:

Event sources maintain the event source mapping, except for the poll-based services (Amazon Kinesis Data Streams, Amazon DynamoDB Streams and Amazon Simple Queue Service). For the poll-based services, AWS Lambda maintains the event source mapping.

https://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-function.html

For other sources, the console itself appears to be making "discovery" API calls to try to piece these things together to present them to the user.

And this appears to be a bug in that logic.

Something is parsing the function policy and incorrectly treating an ArnLike condition as a literal string match. Essentially a cosmetic error, since as was mentioned in comments, the policy does seem to work as intended.

And, this wildcard policy is probably the best way to accomplish the intended objective, although you might want to be more specific, using a string prefix before the *. Whether this is necessary depends on how many users and/or roles are allowed to create these events, and whether you need finer-grained permissions control.

It should, alternately, be possible to do something like this:

Condition": {
    "ArnLike": {
      "AWS:SourceArn": [
        "arn:aws:events<some_arn_id>:rule/my_rule_1",
        "arn:aws:events<some_arn_id>:rule/my_rule_2",
        "arn:aws:events<some_arn_id>:rule/my_rule_3"
        ]
    }
}

But this still doesn't scale as well as a wildcard, and without storing these ARNs externally, you could easily automate a misconfiguration and "misplace" an ARN if your application caused conflicting writes to the policy document.