How to add my own log to log rotation/S3 backup on Amazon Elastic Beanstalk?

Question

We have a PHP application that lives on Amazon Elastic Beanstalk. The application has log rotation and backup to S3 enabled. Apache access and error logs do get properly rotated and backed up every hour.

However the application also creates its own log file. I want to do the same thing with it - every hour it should be rotated and backed up to S3. Following the instructions here I created the following file:

.ebextensions/publish-logs.config

files: 
  "/opt/elasticbeanstalk/tasks/publishlogs.d/cloud-init.conf" :
    mode: "000755"
    owner: root
    group: root
    content: |
      /var/app/current/log/*.log

Then I uploaded the new version to Amazon.

The results - I see that the log file was backed up to S3 ONCE at the very first rotation. And it was not gzipped, just copied. After that, nothing. No new backups to S3. No rotation. When downloading bundle logs, the file is there, in its full glory of about 80MB now (accumulated over several days).

Amazon's documentation is pretty sparse. But it does say that:

When you configure your application's log files for log rotation, the application doesn't need to create copies of log files. Elastic Beanstalk configures logrotate to make a copy of your application's log files for each rotation.

What have I done wrong?

Answer 1

To do that you need to configure logrotate, and that is a bit tricky. Since highly depends on the instance you are using. But let me try. Add this two files to your configuration. First creates configuration for logrotate, and second configures cron to run logrotate with that configuration.

files:
    "/etc/logrotate.d/logrotate.elasticbeanstalk.php.conf":
        mode: "000655"
        owner: root
        group: root
        content: |
            /var/app/current/log/*.log {
                rotate 14
                size 100M
                daily
                compress
                delaycompress
            }
    "/etc/cron.daily/cron.logrotate.elasticbeanstalk.php.conf":
        mode: "000655"
        owner: root
        group: root
        content: |
            #!/bin/sh
            test -x /usr/sbin/logrotate || exit 0
            /usr/sbin/logrotate /etc/logrotate.d/logrotate.elasticbeanstalk.php.conf
            /sbin/service awslogs restart

Give it a try. If fail - please provide AMI ID you are using