Logging stdout with python3 and uWSGI

Question

I have a python 2 flask app running behind uWSGI, which is managed by supervisord. Logs are being written to sys.stdout by the flask app. These logs are then picked up by uWSGI and written to file by supervisord. uWSGI logs are written to /dev/stderr. See the supervisord conf below.

[program:uwsgi]
command = uwsgi --ini /etc/uwsgi.conf --master
directory = /app
autostart = true
autorestart = true
stdout_logfile = /var/log/myapplication/application.log
stdout_logfile_maxbytes = 50000000
stdout_logfile_backups = 3
stderr_logfile = /dev/stderr
stderr_logfile_maxbytes = 0
stopsignal = INT

This works great! I have nicely separated application logs and uWSGI logs.

I've now upgraded to python 3. This has all gone fine, except the application logs are now ending up in uWSGI's stderr, mixed in with the uWSGI logs.

I've trawled the uWSGI docks, and have not been able to find a reason for this change between python 2 and python 3.

I've tried redirecting when defining the socket, like is suggested here http://lists.unbit.it/pipermail/uwsgi/2016-February/008383.html but that just redirects everything (application and uWSGI logs) to /dev/null.

I also found this http://lists.unbit.it/pipermail/uwsgi/2016-January/008353.html but could't find anything about pyimport-shared.

Does anyone know what might be going on?

My uWSGI conf is here if it help.

[uwsgi]
uid = www-data
gid = www-data
module = application:application

socket = /run/uwsgi.sock

single-interpreter = true
enable-threads = true
buffer-size = 16384

processes = 4

Thanks

Answer 1

As you are running 4 processes, I guess you are running uWsgi with --master flag. In this case, you may want to delegate the login process to the master process by using the flag

--log-master

Somehow this solved the problem for me.

from https://github.com/unbit/uwsgi/blob/d960f8fdc36b33c60b7291ca8e32dbb5df22cf15/core/uwsgi.c#L794

{"log-master", no_argument, 0, "delegate logging to master process", uwsgi_opt_true, &uwsgi.log_master, UWSGI_OPT_MASTER|UWSGI_OPT_LOG_MASTER},

Another option:

From https://github.com/unbit/uwsgi/issues/1601

if you need to split stdout and stderr, just remap their file descriptors using python code executed on startup. They (fd: 1 and 2) point to the same resource, but if you are forced to supervisord and need to split them, the only solution is closing and reopening the related files.

Take into account that you have the 'python' logger too, if instead of using unix file descriptor you prefer the python logging subsystem.

Or, well you could remove supervisord from the stack :)