How do I retain apache logs in EC2 when using AWS auto-scaling

Question

We have built a website using AWS EC2 and auto-scaling in a typical LAMP stack (ubuntu).

Scaling etc works well, however, since the instances are "temporary" our apache logs are not retained (as we do not retain the volumes or instances) after load spikes.

Is there a "best practise / most reliable" way to retain our apache logs for these instances?

One idea was to copy log files to S3, during shutdown, by writing a bash script to execute using the /etc/rc0.d functionality (running a script on shutdown).

Answer 1

The "best practice" would be to aggregate all your logs on a server that allows you to archive and search them. You could back up older logs in S3 and eventually Glacier.

To make all of this work, you'll need to set up apache to write hourly or minutely logs and write a cronjob to rsync them to some central place or upload them to S3.

Check out http://logstash.net/ for an open-source aggregation & search solution you could run on your own instance and http://loggly.com/ for a non-free fully-hosted solution.

EDIT: My first thought was "don't do it on shutdown." You'll want to sync your log files as regularly as possible and as a result (if you do minutely logs, for example) you get "near realtime" log aggregation/backup and search.