AWS CloudFormation Events Api Connection

Question

I am trying to create a cloud formation stack using AWS Events to trigger an API call on a schedule. Most of the stack is working, however, the AWS::Events::ApiConnection is failing to create and I am not sure why.

This is the CF snippet that is failing: (Note, The API doesn't have any authentication yet, however, cloud formation requires the AuthParameters property)

"CronServerApiConnection": {
      "Type": "AWS::Events::Connection",
      "Properties": {
        "Name": "api-connection",
        "AuthorizationType": "API_KEY",
        "AuthParameters": {
          "ApiKeyAuthParameters": {
            "ApiKeyName": "foo",
            "ApiKeyValue": "bar"
          }
        }
      }
    },

In the cloud formation console this fails to create with the following error:

Resource handler returned message: "Error occurred during operation 'AWS::Events::Connection'." (RequestToken: xxxxxxxxxxxxxxxxx, HandlerErrorCode: GeneralServiceException)

I can't for the life of me figure this one out. from what I can see my CF snippet matches exactly what AWS specify in their docs here https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-connection.html

Answer 1

I ran into this issue myself a few weeks ago, and while looking for an answer I found this question unresolved so I thought I would share the answer. The events API is not descriptive at all with any of the errors, in my case the issues were permissions related. While is not clear in the documentation the AWS::Events::Connection not only needs permissions for the events API but also for the secretsmanager API since it will create some secrets for you under the hood. I solved this by adding full API permissions to the role creating the stack but of course I scoped the permissions by the resource to avoid security issues, something like:

effects: "Allow"
actions: [
        "events:*",
        "secretsmanager:*"
      ]
resources: [
        "arn:aws:secretsmanager:<your region>:<your-account-id>:secret:events!connection/<yoursecretnameprefix>-*"
      ]

I will leave the addition of the event resource to you, but essentially is the same just scope by the arn of your resource. The above is just an example please replace the placeholders with the correct values.

Answer 2

I fought this issue last night. I think the general exception is generating the multiple types of responses and solutions being offered. In my case, it was something different that was already posted. Hopefully , this helps someone else.

My Scenario:

• Using the AWS CDK I have a ServiceStack for a backend API
• The Stack uses a pre-existing EventBus to create Rules, Connections, and ApiDestinations that point to itself. (It is interested in events from that bus so it "self registers" its own rules)
• The Stack also sets up the DNS, ACM Cert, ALB and ECS Service for that backend service

After much run running around, what I ended up suspecting was that the error was somehow related to DNS or maybe because the service wasn’t “live” yet.

I tested my hypothesis by essentially, removing the ApiDestinations from the Stack, destroying everything and deploying it again. Boom, everything worked. I then added the ApiDestinations and boom, it worked. So now I realized that the ApiDestination creation is doing some validation against the URL.

I manually validated this logic on the AWS Console:

• If you create an ApiDestination, say to https://www.google.com, it works.
• If you create an ApiDestination, to say https://domain.that.doesnotexist, it throws an error
• If you create a DNS Record with Valid Cert manually on your domain, but it doesn’t point to anything that returns an HTTP response, it also throws the error.
• If you create an ApiDestination with a “live” URL where it’s HTTPS but the cert is invalid, it is ok with this. (Meaning, cert validity doesn’t seem to matter as long as the URL starts with HTTPS)

So essentially, for me it was a chicken egg problem because I have everything in one Stack. Probably not the best solution but what I did was just add a manual dependency to the ApiDestinations myApiDestination.node.addDepedency(myServiceARecord). Inherently, myServiceRecord points to my alb.dnsName so there is that dependency on that being up. Sigh. Perhaps the better solution is to break it up and have the EventBridge stuff in a separate Stack.

You may have a yet different solution but the point here is that when you create an ApiDestination, the endpoint URL must start with HTTPS, resolve and actually responds with any HTTP response.

In my case, I was able to get away with the ALB’s 502 “Service Unavailable” response because my ECS service was not up yet at the time CloudFormation created the ApiDestinations.