Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Issue with awslogs service and CloudWatch Logs Agent on Ubuntu 16.04

On one of my AWS ec2 instances running Ubuntu 16.04, I'm getting the following errors filled up in my /var/syslog.

Jul 17 18:11:21 Mysql-Slave systemd[1]: Stopped The CloudWatch Logs agent.
Jul 17 18:11:21 Mysql-Slave systemd[1]: Started The CloudWatch Logs agent.
Jul 17 18:11:26 Mysql-Slave systemd[1]: awslogs.service: Main process exited, code=exited, status=255/n/a
Jul 17 18:11:26 Mysql-Slave systemd[1]: awslogs.service: Unit entered failed state.
Jul 17 18:11:26 Mysql-Slave systemd[1]: awslogs.service: Failed with result 'exit-code'.
Jul 17 18:11:26 Mysql-Slave systemd[1]: awslogs.service: Service hold-off time over, scheduling restart.
Jul 17 18:11:26 Mysql-Slave systemd[1]: Stopped The CloudWatch Logs agent.
Jul 17 18:11:26 Mysql-Slave systemd[1]: Started The CloudWatch Logs agent.
Jul 17 18:11:32 Mysql-Slave systemd[1]: awslogs.service: Main process exited, code=exited, status=255/n/a
Jul 17 18:11:32 Mysql-Slave systemd[1]: awslogs.service: Unit entered failed state.
Jul 17 18:11:32 Mysql-Slave systemd[1]: awslogs.service: Failed with result 'exit-code'.
Jul 17 18:11:32 Mysql-Slave systemd[1]: awslogs.service: Service hold-off time over, scheduling restart.
Jul 17 18:11:32 Mysql-Slave systemd[1]: Stopped The CloudWatch Logs agent.
Jul 17 18:11:32 Mysql-Slave systemd[1]: Started The CloudWatch Logs agent.

The /var/log/awslogs.log contains these messages:

database is locked
2018-07-17 20:59:01,055 - cwlogs.push - INFO - 27074 - MainThread - Missing or invalid value for use_gzip_http_content_encoding config. Defaulting to using gzip encoding.
2018-07-17 20:59:01,055 - cwlogs.push - INFO - 27074 - MainThread - Using default logging configuration.

database is locked
2018-07-17 20:59:06,549 - cwlogs.push - INFO - 27104 - MainThread - Missing or invalid value for use_gzip_http_content_encoding config. Defaulting to using gzip encoding.
2018-07-17 20:59:06,549 - cwlogs.push - INFO - 27104 - MainThread - Using default logging configuration.

database is locked
2018-07-17 20:59:12,054 - cwlogs.push - INFO - 27110 - MainThread - Missing or invalid value for use_gzip_http_content_encoding config. Defaulting to using gzip encoding.
2018-07-17 20:59:12,054 - cwlogs.push - INFO - 27110 - MainThread - Using default logging configuration.

Any pointers in troubleshooting this will be of great help.

like image 659
Deb Avatar asked Oct 19 '25 03:10

Deb


2 Answers

A similar issue was posted in the following link - https://forums.aws.amazon.com/thread.jspa?threadID=165134

I did the following:

a) Stopped the awslogs service

$ service awslogs stop  ## Amazon Linux

OR

$ service awslogsd stop  ## Amazon Linux 2

b) Deleted the agent-state file in /var/awslogs/state/ (I renamed it in my case)

$  mv agent-state agent-state.old  ## Amazon Linux

OR

$ cd /var/lib/awslogs; mv agent-stat agent-stat.old  ## Amazon Linux 2

c) Restarted the awslogs service

$ service awslogs start  ## Amazon Linux

OR

$ sudo systemctl start awslogsd  ## Amazon Linux 2

A new agent-state file was created as a result and the errors mentioned my post disappeared after this.

like image 121
Deb Avatar answered Oct 21 '25 16:10

Deb


Please try the following commands based on your Linux version

sudo service awslogs start

If you are running Amazon Linux 2, try the below command sudo systemctl start awslogsd took me 2 hours to figure this out

like image 43
Praba Avatar answered Oct 21 '25 17:10

Praba



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!