CloudWatch resource access policy error while creating Amazon Elasticsearch Service via Cloud Formation

Question

I am trying to create an elastic search domain with enabled LogPublishingOptions. While enabling LogPublishingOptions ES says it does not sufficient permissions to create a LogStream on Cloudwatch.

I tried creating a policy with a role and attaching the policy to the LogGroup which is referred by ES but it ain't working. Following is my elastic search cloud formation template,

AWSTemplateFormatVersion: 2010-09-09

Resources:
  MYLOGGROUP:
    Type: 'AWS::Logs::LogGroup'
    Properties:
      LogGroupName: index_slow

  MYESROLE:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: es.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/AmazonESFullAccess'
        - 'arn:aws:iam::aws:policy/CloudWatchFullAccess'
      RoleName: !Join
        - '-'
        - - es
          - !Ref 'AWS::Region'

  PolicyDocESIndexSlow :
    Type: AWS::IAM::Policy
    Properties:
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: 
             - logs:PutLogEvents
             - logs:CreateLogStream
            Resource: 'arn:aws:logs:*'
      PolicyName: !Ref MYLOGGROUP
      Roles:
        - !Ref MYESROLE

  MYESDOMAIN:
    Type: AWS::Elasticsearch::Domain
    Properties:
      DomainName: 'es-domain'
      ElasticsearchVersion: '7.4'
      ElasticsearchClusterConfig:
        DedicatedMasterCount: 3
        DedicatedMasterEnabled: True
        DedicatedMasterType: 'r5.large.elasticsearch'
        InstanceCount: '2'
        InstanceType: 'r5.large.elasticsearch'
      EBSOptions:
        EBSEnabled: True
        VolumeSize: 10
        VolumeType: 'gp2'
      AccessPolicies:
        Version: 2012-10-17
        Statement:
          - Effect: Deny
            Principal:
              AWS: '*'
            Action: 'es:*'
            Resource: '*'
      AdvancedOptions:
        rest.action.multi.allow_explicit_index: True
      LogPublishingOptions:
        INDEX_SLOW_LOGS:
          CloudWatchLogsLogGroupArn: !GetAtt
            - MYLOGGROUP
            - Arn
          Enabled: True
      VPCOptions:
        SubnetIds:
          - !Ref MYSUBNET
        SecurityGroupIds:
          - !Ref MYSECURITYGROUP
  MYVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
  MYSUBNET:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MYVPC
      CidrBlock: 10.0.0.0/16
  MYSECURITYGROUP:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: security group for elastic search domain
      VpcId: !Ref MYVPC
      GroupName: 'SG for ES'
      SecurityGroupIngress:
        - FromPort: '443'
          IpProtocol: tcp
          ToPort: '443'
          CidrIp: 0.0.0.0/0

Upon execution, it creates all resources except MYESDOMAIN. It says

The Resource Access Policy specified for the CloudWatch Logs log group index_slow does not grant sufficient permissions for Amazon Elasticsearch Service to create a log stream. Please check the Resource Access Policy. (Service: AWSElasticsearch; Status Code: 400; Error Code: ValidationException)

Any idea what's missing here?

Answer 1

Update 2021

There is a CloudFormation resource called AWS::Logs::ResourcePolicy which allows defining policies for CloudWatch Logs in CF. The main issue I found is that it only accepts a real string as the value. Trying to assemble a string using Ref, Join, etc kept being rejected. If anyone can make that work that would be fab.

Writing it in YAML is easier as JSON requires escaping all the " characters.

OSLogGroupPolicy:
    Type: AWS::Logs::ResourcePolicy
    Properties:
      PolicyName: AllowES
      PolicyDocument: '{"Version": "2012-10-17","Statement":[{"Effect":"Allow","Principal": {"Service": ["es.amazonaws.com"]},"Action":["logs:PutLogEvents","logs:CreateLogStream"],"Resource":"*"}]}'

Answer 2

I believe there is some confusion here about what policies should be updated/set to enable ES writing to a log group.

I think you should apply the PolicyDocESIndexSlow policy to CloudWatch Logs.

And this can't be done in CloudFormation from what I remember. You have to use put-resource-policy, corresponding API call, or console as shown in:

• Viewing Amazon Elasticsearch Service Slow Logs