Menu
How to add custom logs to CloudWatch? Defaults logs are sent but how to add a custom one?
I already added a file like this: (in .ebextensions)
files: "/opt/elasticbeanstalk/tasks/bundlelogs.d/applogs.conf" : mode: "000755" owner: root group: root content: | /var/app/current/logs/* "/opt/elasticbeanstalk/tasks/taillogs.d/cloud-init.conf" : mode: "000755" owner: root group: root content: | /var/app/current/logs/* As I did bundlelogs.d and taillogs.d these custom logs are now tailed or retrieved from the console or web, that's nice but they don't persist and are not sent on CloudWatch.
In CloudWatch I have the defaults logs like/aws/elasticbeanstalk/InstanceName/var/log/eb-activity.log
And I want to have another one like this/aws/elasticbeanstalk/InstanceName/var/app/current/logs/mycustomlog.log
863
Go to the AWS CloudWatch console and click on Logs at the left most; select the CloudTrail Log group that we just created earlier, and click on Actions and select Stream to Amazon Elasticsearch Service.
To stream instance logs to CloudWatch LogsOpen the Elastic Beanstalk console , and in the Regions list, select your AWS Region. In the navigation pane, choose Environments, and then choose the name of your environment from the list. If you have many environments, use the search bar to filter the environment list.
Elastic Beanstalk automatically uses Amazon CloudWatch to help you monitor your application and environment status. You can navigate to the Amazon CloudWatch console to see your dashboard and get an overview of all of your resources as well as your alarms. You can also choose to view more metrics or add custom metrics.
This article details the steps required to include your custom application logs from ElasticBeanstalk provisioned EC2 instances to the CloudWatch Logs. Step 1. Set up the IAM Policy & role Step 2. Determine your application log paths and files Step 3. Configure log streaming to CloudWatch
AWS provide a mechanism to extend the functionality of Elastic Beanstalk via a special .ebextensions folder in your project. They are evaluated at least every time an instance is created, which is an ideal time to configure your custom application logging requirements.
You can use the AWS… According to the AWS Beanstalk documentation, each application and web server stores logs in its own folder: However this isn’t strictly accurate, for example my Apache Tomcat logs were not in /var/log/tomcat8/ but /var/log/tomcat/ (note the version number ‘8’ is excluded).
Update your Gradle Script to include the ebextension in the root of the file With this gradle script, your war file should have a .ebextensions folder in the root and should have the mycustom.conf file in it. Now let’s prepare your Elastic Beanstalk to enable the cloudwatch
Both bundlelogs.d and taillogs.d are logs retrieved from management console. What you want to do is extend default logs (e.g. eb-activity.log) to CloudWatch Logs. In order to extend the log stream, you need to add another configuration under /etc/awslogs/config/. The configuration should follow the Agent Configuration file Format.
I've successfully extended my logs for my custom ubuntu/nginx/php platform. Here is my extension file FYI. Here is an official sample FYI.
In your case, it could be like
files: "/etc/awslogs/config/my_app_log.conf" : mode: "000600" owner: root group: root content: | [/var/app/current/logs/xxx.log] log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/app/current/logs/xxx.log"]]}` log_stream_name = {instance_id} file = /var/app/current/logs/xxx.log* Credits where due go to Sebastian Hsu and Abhyudit Jain.
This is the final config file I came up with for .ebextensions for our particular use case. Notes explaining some aspects are below the code block.
files: "/etc/awslogs/config/beanstalklogs_custom.conf" : mode: "000600" owner: root group: root content: | [/var/log/tomcat8/catalina.out] log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Fn::Select" : [ "1", { "Fn::Split" : [ "-", { "Ref":"AWSEBEnvironmentName" } ] } ] }, "var/log/tomcat8/catalina.out"]]}` log_stream_name = `{"Fn::Join":["--", [{ "Ref":"AWSEBEnvironmentName" }, "{instance_id}"]]}` file = /var/log/tomcat8/catalina.out* services: sysvinit: awslogs: files: - "/etc/awslogs/config/beanstalklogs_custom.conf" commands: rm_beanstalklogs_custom_bak: command: "rm beanstalklogs_custom.conf.bak" cwd: "/etc/awslogs/config" ignoreErrors: true We have a standard naming scheme for our EB environments which is exactly environmentName-environmentType. I'm using { "Fn::Split" : [ "-", { "Ref":"AWSEBEnvironmentName" } ] } to split that into an array of two strings (name and type).
Then I use { "Fn::Select" : [ "1", <<SPLIT_OUTPUT>> ] } to get just the type string. Your needs would obviously differ, so you may only need the following:
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat8/catalina.out"]]}` I'm using the Fn::Join function to join the EB environment name with the instance ID. Note that the instance ID template is a string that gets echoed exactly as given.
The awslogs service is restarted automatically when the custom conf file is deployed.
When the files block overwrites an existing file, it creates a backup file, like beanstalklogs_custom.conf.bak. This block erases that backup file because awslogs service reads both files, potentially causing conflict.
If you log in to an EC2 instance and sudo cat the file, you should see something like this. Note that all the Fn functions have resolved. If you find that an Fn function didn't resolve, check it for syntax errors.
[/var/log/tomcat8/catalina.out] log_group_name = /aws/elasticbeanstalk/environmentType/var/log/tomcat8/catalina.out log_stream_name = environmentName-environmentType--{instance_id} file = /var/log/tomcat8/catalina.out* If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
