Lambda backed custom resource cf template returns 'CREATE_FAILED'

Question

The below lambda function is to associate a SNS topic to the existing directories, followed by a custom resource to invoke the lambda func itself. I see that the lambda creation is successful with the 'Register_event_topic' also completing. However, the stack fails after a while mostly because the 'custom resource failed to stabilize in expected time'; How can I ensure that the stack does not error out?

AWSTemplateFormatVersion: '2010-09-09'
    #creating lambda function to register_event_topic
    Description: Lambda function to register event topic with existing directory ID
    Parameters:
      RoleName:
        Type: String
        Description: "IAM Role used for Lambda execution"
        Default: "arn:aws:iam::<<Accountnumber>>:role/LambdaExecutionRole"
      EnvVariable:
        Type: String
        Description: "The Environment variable set for the lambda func"
        Default: "ESdirsvcSNS"
    Resources:
      REGISTEREVENTTOPIC:
        Type: 'AWS::Lambda::Function'
        Properties:
          FunctionName: dirsvc_snstopic_lambda
          Handler: index.lambda_handler
          Runtime: python3.6
          Description: Lambda func code to assoc dirID with created SNS topic
          Code:
            ZipFile: |
              import boto3
              import os
              import logging
              dsclient = boto3.client('ds')
              def lambda_handler(event, context):
                response = dsclient.describe_directories()
                directoryList = []
                print(response)
                for directoryList in response['DirectoryDescriptions']:
                    listTopics = dsclient.describe_event_topics(
                      DirectoryId=directoryList['DirectoryId']
                    )
                    eventTopics = listTopics['EventTopics']
                    topiclength = len(eventTopics)
                    if topiclength == 0:
                      response = dsclient.register_event_topic(
                          DirectoryId=directoryList['DirectoryId'],
                          TopicName= (os.environ['MONITORING_TOPIC_NAME'])
                      )  
                    print(listTopics)
          Timeout: 60
          Environment:
            Variables:
              MONITORING_TOPIC_NAME: !Ref EnvVariable
          Role: !Ref RoleName

      InvokeLambda:
        Type: Custom::InvokeLambda
        Properties:
          ServiceToken: !GetAtt REGISTEREVENTTOPIC.Arn
          ReservedConcurrentExecutions: 1

Answer 1

Alas, writing a Custom Resource is not as simple as you'd initially think. Instead, special code must be added to post the response back to a URL.

You can see this in the sample Zip file provided on: Walkthrough: Looking Up Amazon Machine Image IDs - AWS CloudFormation

From the Custom Resources - AWS CloudFormation documentation:

The custom resource provider processes the AWS CloudFormation request and returns a response of SUCCESS or FAILED to the pre-signed URL. The custom resource provider provides the response in a JSON-formatted file and uploads it to the pre-signed S3 URL.

This is due to the asynchronous behaviour of CloudFormation. It doesn't simply call the Lambda function and then wait for a response. Rather, it triggers the Lambda function and the function must call back and trigger the next step in CloudFormation.

Answer 2

Your lambda doesn't support custom resource life cycle

In a Lambda backed custom resource, you implement your logic to support creation, update and deletion of the resource. These indications are sent from CloudFormation via the event and give you information about the stack process.

In addition, you should also return your status back to CloudFormation

CloudFormation expects to get a response from your Lambda function after you're done with your logic. It will not continue with the deployment process if it doesn’t get a response, or at least until a 1 hour(!) timeout is reached. It can cost you a lot of time and frustration.

You can read more here