Elastic Beanstalk Docker with Amazon CloudWatch

Question

I have an Elastic Beanstalk application deployed with a Docker container. The application itself is a Java Application.

My goal is to get the logs to Cloudwatch. In particular I would like to get the stdouterr.log file to Cloudwatch. The file can be found under /var/log/eb-docker/containers/eb-current-app/*

I followed the official AWS documentation here. Based on the example configuration files I managed to get the nginx Webrequest to Cloudwatch.

For the EB docker stdouterr log I adapted the cwl-log-setup.config file to the following:

Mappings:
  CWLogs:
    ApplicationLogGroup:
      LogFile: "/var/log/eb-docker/containers/eb-current-app/*"
      TimestampFormat: "%d/%b/%Y:%H:%M:%S %z"

Outputs:
  ApplicationLogGroup:
    Description: "The name of the Cloudwatch Logs Log Group created for this environments web server access logs. You can specify this by setting the value for the environment variable: WebRequestCWLogGroup. Please note: if you update this value, then you will need to go and clear out the old cloudwatch logs group and delete it through Cloudwatch Logs."
    Value: { "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup"}


Resources :
  AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup:    ## Must have prefix:  AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0
    Type: "AWS::Logs::LogGroup"
    DependsOn: AWSEBBeanstalkMetadata
    DeletionPolicy: Retain     ## this is required
    Properties:
      LogGroupName:
        "Fn::GetOptionSetting":
          Namespace: "aws:elasticbeanstalk:application:environment"
          OptionName: ApplicationLogGroup
          DefaultValue: {"Fn::Join":["-", [{ "Ref":"AWSEBEnvironmentName" }, "webrequests"]]}
      RetentionInDays: 14

The cloudwatch log group is created but no logs arrive. What steps am I missing or what is wrong in my configuration file?

Answer 1

I create Dockerrun file for each ElasticBeanstalk application. https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html

There you can specify a logging configuration:

           "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "${you-log-group-name}",
                    "awslogs-region": "${region}",
                    "awslogs-stream-prefix": "${serviceName}"
                }
            }

This sets up Docker's logging driver configuration more details: https://docs.docker.com/config/containers/logging/configure/

Answer 2

I've just encountered the same issue - I managed to get the log files by changing the LogFile configuration to

Mappings:
  CWLogs:
    WebRequestLogGroup:
      LogFile: "/var/log/eb-docker/containers/eb-current-app/*.log"
      TimestampFormat: "%d/%b/%Y:%H:%M:%S %z"

Note this only works if there is a single log file, if you re-deploy a container or apply a configuration change which results in multiple logs in this directory then only the events from the log file with the most current modified time will be processed by the awslogs agent

Also by default the agent will compare the first line of the log file to determine if it is a different file, if the first line is the same it will ignore it. You can specify the lines the agent uses to fingerprint the file by adding file_fingerprint_lines configuration,

for example to use lines 1 - 20 to identify the file:

AWSEBAutoScalingGroup:
    Metadata:
      "AWS::CloudFormation::Init":
        CWLogsAgentConfigSetup:
          files:
            ## any .conf file put into /tmp/cwlogs/conf.d will be added to the cwlogs config (see cwl-agent.config)
            "/tmp/cwlogs/conf.d/apache-access.conf":
              content : |
            [    apache-access_log]
                file = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "LogFile"]}`
                log_group_name = `{ "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup" }`
                file_fingerprint_lines = 1-20
                log_stream_name = {instance_id}
                datetime_format = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "TimestampFormat"]}`
              mode  : "000400"
              owner : root
              group : root