AWS Elastic Beanstalk logging with python (django)

Question

How do you manage your application logs in AWS elastic beanstalk? Which file you write you application logs to?

I'm using the following Logging configuration in my development environment but this doesn't work when I deploy in AWS.

DEBUG_LOG_DIR = BASE_DIR + "/django_debug.log" LOGGING = {     'version': 1,     'disable_existing_loggers': True,     # How to format the output     'formatters': {         'standard': {             'format' : "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",             'datefmt' : "%d/%b/%Y %H:%M:%S"         },     },     # Log handlers (where to go)     'handlers': {         'null': {             'level':'DEBUG',             'class':'django.utils.log.NullHandler',         },         'log_file': {             'level':'DEBUG',             'class':'logging.handlers.RotatingFileHandler',             'filename': DEBUG_LOG_DIR,             'maxBytes': 50000,             'backupCount': 2,             'formatter': 'standard',         },         'console':{             'level':'INFO',             'class':'logging.StreamHandler',             'formatter': 'standard'         },         'mail_admins': {             'level': 'ERROR',             'class': 'django.utils.log.AdminEmailHandler',         },     },     # Loggers (where does the log come from)     'loggers': {         'repackager': {             'handlers': ['console', 'log_file'],             'level': 'DEBUG',             'propagate': True,         },         'django': {             'handlers':['console'],             'propagate': True,             'level':'WARN',         },         'django.db.backends': {             'handlers': ['console', 'log_file'],             'level': 'WARN',             'propagate': False,         },         '': {             'handlers': ['console', 'log_file'],             'level': 'DEBUG',         },     } } 

Answer 1

I had a similar issue but on Elastic Beanstalk, so I created a config file (e.g. applogs.config) in .ebextensions folder of the app. This creates the app-logs folder if it is not there already and sets the file permissions and owner so that the app can write its logs there.

commands:   00_create_dir:     command: mkdir -p /var/log/app-logs   01_change_permissions:     command: chmod g+s /var/log/app-logs   02_change_owner:     command: chown wsgi:wsgi /var/log/app-logs 

Finally, in your Django settings:

LOGGING = {     'version': 1,     'disable_existing_loggers': False,     'handlers': {         'file': {             'level': 'DEBUG',             'class': 'logging.FileHandler',             'filename': '/var/log/app-logs/django.log',         },     },     'loggers': {         'django': {             'handlers': ['file'],             'level': 'DEBUG',             'propagate': True,         },     }, } 

Aditionally, if you want your log to be accessible from beanstalk logs using the web, add this to your file in .ebextensions

files:   "/opt/elasticbeanstalk/tasks/taillogs.d/django.conf":     mode: "000755"     owner: root     group: root     content: |       /var/log/app-logs/django.log 

Answer 2

Ok, I figured out a way to do it.

First I connected via ssh to ec2 machine, then I create a folder in /var/log called app_logs with root user:

mkdir /var/log/app_logs 

After that I did the follow:

cd /var/log/ chmod g+s app_logs/ setfacl -d -m g::rw app_logs/ chown wsgi:wsgi app_logs/ 

That ensures that all the files created in this folder will have wsgi as owner and will be writable for the group that the file belongs. I had to do that because I noticed that the log file created by django app had root as owner and owner group but the application runs through wsgi user.

Finally I changed DEBUG_LOG_DIR to /var/log/app_logs/django_debug.log