uWSGI / Flask / Python logs stop after some time

Question

I have a uWSGI / Flask setup using python loggers. Though logs only from some workers get to the logs and after some time even those cease to show up at all. My hypothesis is that when uWSGI restarts (clones) workers, logging somehow gets broken. Any ideas?

app/server.py:

app = Flask(...)
handler = logging.StreamHandler()
app.logger.addHandler(handler)
app.run()

uWSGI:

uwsgi --emperor /etc/uwsgi/apps-enabled/*.ini --die-on-term --uid www-data --gid www-data --logto /var/www/app.com/logs/uwsgi/emperor.log --socket /tmp/uwsgi/emperor.sock --enable-threads --master --single-interpreter --log-reopen --chmod-socket=770

apps-enabled/app-0.ini and apps-enabled/app-1.ini look like this:

module=server:app
enable-threads=true
single-interpreter=true
master=true
chdir=/var/www/app.com/app
env=APPLICATION_ENVIRONMENT=production
venv=/var/www/app.com/virtualenv

logto=/var/www/app.com/logs/uwsgi/app.com-0.log
log-reopen=true
chmod-socket=770
buffer-size=65535

lazy-apps=true
max-requests=5000
heartbeat=15

for=0 1 2 3 4 5 6 7
socket=/tmp/uwsgi/app.0.%(_).sock
endfor=

processes=8

map-socket=0:1
map-socket=1:2
map-socket=2:3
map-socket=3:4
map-socket=4:5
map=socket=5:6
map=socket=6:7
map=socket=7:8

I have also tried to use SysLogHandler with the same result.

Answer 1

I'm not sure that this will help in your case, but you can try to use post forking in uWSGI.

The postfork decorator is just the ticket. You can declare multiple postfork tasks. Each decorated function will be executed in sequence after each fork().

@postfork
def init_logging():
    app.logger.addHandler(handler)

Or you can specify lazy-apps=true in your uwsgi.ini.

For details: http://uwsgi-docs.readthedocs.io/en/latest/PythonDecorators.html and http://uwsgi-docs.readthedocs.io/en/latest/articles/TheArtOfGracefulReloading.html