Cloudformation template to push cloudwatch logs to elasticsearch

Question

I am looking for a Cloudformation template to push cloudwatch logs to elasticsearch in another account. Even to the same account would be ok and I can update that.

Seems like a standard problem but haven't seen any template which automates the steps described in https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_ES_Stream.html.

Answer 1

I should have updated the answer. This is what I wrote which is the minimalistic version of streaming logs from CW to Elastic Search.

---

  Resources:
      LambdaElasticSearchExecutionRole:
          Properties:
            AssumeRolePolicyDocument:
              Statement:
              - Action: ['sts:AssumeRole']
                Effect: Allow
                Principal:
                  Service: [lambda.amazonaws.com]
              Version: '2012-10-17'
            Policies:
            - PolicyDocument:
                Statement:
                - Action: ['es:ESHttpPost']
                  Effect: Allow
                  Resource: "*"
                - Action: ['logs:CreateLogGroup', 'logs:CreateLogStream', 'logs:PutLogEvents']
                  Effect: Allow
                  Resource: "arn:aws:logs:*:*:*"
                - Action: ['lambda:InvokeFunction']
                  Effect: Allow
                  Resource: "arn:aws:logs:*:*:*"
                Version: '2012-10-17'
              PolicyName: lambdaRoleElasticSearchStreaming
          Type: AWS::IAM::Role

      ESStreamingLambda:
         Type: AWS::Lambda::Function
         DependsOn: LambdaElasticSearchExecutionRole
         Properties:
           Handler: index.handler
           Role:
             Fn::GetAtt: [LambdaElasticSearchExecutionRole, Arn]
           Code:
             S3Bucket: {'Fn::Sub': 'do-not-delete-cw-es-log-streaming-lamda-${AWS::Region}'}
             S3Key: LogsToElasticsearch.zip
           Runtime: nodejs4.3

      LambdaPermissionForCWInvokation:
         DependsOn: ESStreamingLambda
         Type: AWS::Lambda::Permission
         Properties:
          Action: lambda:InvokeFunction
          FunctionName:
            Fn::GetAtt: [ESStreamingLambda, Arn]
          Principal: {'Fn::Sub': 'logs.${AWS::Region}.amazonaws.com'}
  APILogsToElasticSearchSubscriptionFilter:
    Type: AWS::Logs::SubscriptionFilter
    DependsOn: [ESStreamingLambda, LambdaPermissionForCWInvokation]
    Properties:
      DestinationArn:
        Fn::GetAtt: [ESStreamingLambda, Arn]
      FilterPattern: ''
      LogGroupName: {Ref: LambdaLogGroup}
```java