How can I configure gunicorn to use a consistent error log format?

Question

I am using Gunicorn in front of a Python Flask app. I am able to configure the access log format using the --access-log-format command line parameter when I run gunicorn. But I can't figure out how to configure the error logs.

I would be fine with the default format, except it's not consistent. It looks like Gunicorn status messages have one format, but application exceptions have a different format. This is making it difficult to use log aggregation.

For example, here a few messages from Gunicorn's error log. The first few lines have a different format than the exception line. Event the datetime format is different.

[2017-07-13 16:33:24 +0000] [15] [INFO] Booting worker with pid: 15
[2017-07-13 16:33:24 +0000] [16] [INFO] Booting worker with pid: 16
[2017-07-13 16:33:24 +0000] [17] [INFO] Booting worker with pid: 17
[2017-07-13 16:33:24 +0000] [18] [INFO] Booting worker with pid: 18
[2017-07-13 18:31:11,580] ERROR in app: Exception on /api/users [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
...

What's the best way to configure Gunicorn to use consistent formatting for its error logs?

Answer 1

Using this logging config file, I was able to change the error log format

[loggers]
keys=root, gunicorn.error

[handlers]
keys=error_console

[formatters]
keys=generic

[logger_root]
level=INFO
handlers=error_console

[logger_gunicorn.error]
level=INFO
handlers=error_console
propagate=0
qualname=gunicorn.error

[handler_error_console]
class=StreamHandler
formatter=generic
args=(sys.stderr, )

[formatter_generic]
format=%(asctime)s %(levelname)-5s [%(module)s] ~ %(message)s
datefmt=%Y-%m-%d %H:%M:%S %Z
class=logging.Formatter

The key is to overwrite the gunicorn.error logger config, and the snipped above does exactly that.

Note the propagate=0 field, it is important otherwise your log messages will be printed twice (gunicorn always keeps the default logging config).