Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

some django's logs are missing when host in uwsgi with multiple process

I'm using django+uwsgi for a web project. But I found some django logs would be missing after uwsgi is running for a while!

The situation is that: I config the uwsgi with 8 process. When I start the uwsgi, all the django logs would be written in single log file. But after a few hours, some logs are not written in file. I compared the django log file to uwsgi log file. I found only one uwsgi process's requests were written in django file. The other 7 process's django logs were missing. When I restart the uwsgi, it is the same result.

Does anyone know about this?

Thanks,

my django logging config:

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
    'simple': {
        'format': '%(levelname)s %(asctime)s %(message)s'
    },
    'detail': {
        'format': '%(levelname)s %(asctime)s [%(module)s.%(funcName)s line:%(lineno)d] %(message)s',
    },
},
'handlers': {
    'file': {
        'level': 'INFO',
        'formatter': 'simple',
        'class': 'logging.handlers.TimedRotatingFileHandler',
        'filename': LOG_FILE,
        'when': 'midnight',
        'backupCount': 366,
    },
    'err_file': {
        'level': 'WARN',
        'formatter': 'detail',
        'class': 'logging.handlers.TimedRotatingFileHandler',
        'filename': LOG_ERR_FILE,
        'when': 'midnight',
        'backupCount': 366,
    },
},
'loggers': {
    'django_request': {
        'handlers': ['file', 'err_file'],
        'level': 'DEBUG',
        'propagate': True,
    },
}
}

my uwsgi config:

<uwsgi>
<socket>0.0.0.0:8888</socket>
<chdir>src_dir</chdir>
<pythonpath>..</pythonpath>
<module>wsgi</module>
<workers>4</workers>
<processes>8</processes>
<master>true</master>
<pidfile>uwsgi.pid</pidfile>
<enable-threads>true</enable-threads>
<logdate>true</logdate>
<daemonize>/log/uwsgi/uwsgi.log</daemonize>
</uwsgi>
like image 1000
user1198997 Avatar asked Feb 09 '12 07:02

user1198997


1 Answers

This is because it is not safe to write to the same file from several workers. You should write logs to different files from each worker, or try to use SysLogHandler

http://docs.python.org/library/logging.handlers.html

/etc/rsyslog.d/local-myapp.conf:

local0.*                         -/var/log/myapp/app.log

settings.py:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
        'user': {
            '()': 'core.log.UserFilter',
        }
    },
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(user)s %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(asctime)s %(module)s %(user)s %(message)s'
        },
    },
    'handlers': {
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
            'filters': ['user'],
        },
        'syslog':{
            'level': 'INFO',
            'class': 'logging.handlers.SysLogHandler',
            'formatter': 'verbose',
            'filters': ['user'],
            'facility': 'local0',
            'address': '/dev/log',
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
    },
    'root': {
        'handlers': ['syslog'],
        'level': 'INFO',
    }
}

You also can try to write log messages into console and let uwsgi write the logs by himself.

like image 63
raacer Avatar answered Oct 20 '22 23:10

raacer