Application Logging for elastic beanstalk

Question

I have Spring MVC application running on aws ec2. Now I have decided to move to aws elastic beanstalk and yet exploring it before migrating current application. I had few questions regarding persistent storage and application logging.

My question could be naive but please bear with it :)

I am using log4j for application logging. Log4j appends logs at specified location on hard drive. As elastic beanstalk has no persistent storage (storage will be deleted once instance is terminated) and instances may grow/shrink based on the load rules.

So I am stuck at the following

1. How would I configure log4j for appending the logs.
2. If I were to stick to current log4j implementation, how would I make sure that directory structure be created when an instance is launched( I am assuming no code changes in the application)
3. If I go with socketappender from log4j then how would I distinguish between files from different instances.

I have been going through articles describing use of .ebextensions directory. I tried using basic .conf files provided by AWS to tail logs to S3 but no success for my application logs.

Sample script to create application directory places under .ebextensions

commands:
  01_create_dir:
    test: test ! -d "${DIR}"
    command: mkdir "${DIR}"

Referred links :

• https://aws.amazon.com/blogs/aws/cloudwatch-log-service/
• http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html
• http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environment-configuration-methods-during.html#configuration-options-during-console-ebextensions

EBS console displays apache logs for URL access.

I would like to know if there is any ready-made solution provided by AWS already that I failed to search.

Answer 1

From http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-se-platform.html :

Standard output and error streams from processes started with a Procfile are captured in log files named after the process and stored in /var/log. For example, the web process in the preceding example generates logs named web-1.log and web-1.error.log for stdout and stderr, respectively.

Based on that, all you have to do is ensure log4j / logback outputs the logs you want to stdout, i.e. CONSOLE. With Spring Boot this is the default behavior.