Configuring and using structlog with Django

Question

Does anyone use structlog with Django? I'm looking for a code sample how can I integrate Django logging (which is done via standard library), and structlog.

I've tried the code from the "Rendering Using structlog-based Formatters Within logging" example, with only slightest modifications:

# From my settings.py, basically the same code as in the linked example

timestamper = structlog.processors.TimeStamper(fmt="%Y-%m-%d %H:%M:%S")
pre_chain = [
    structlog.stdlib.add_log_level,
    timestamper,
]
LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": { ... },  # Exactly like in the linked example
    "handlers": { ... },    # Ditto, but only "default" handler (no files)
    "loggers": {
        "django": {
            "handlers": ["default"],
            "level": "INFO",
        },
        # I also had "" logger here, with the same config as "django",
        # but it's irrelevant for the example purposes.
    }
}
# Same as in the example
structlog.configure(
    processors=[
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        timestamper,
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
    ],
    context_class=dict,
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

However, I end up with logging errors. This is an excerpt of what happens on a simple GET request that ends up with 404

TypeError: not all arguments converted during string formatting
...
  File "/usr/local/lib/python3.6/site-packages/django/core/handlers/base.py", line 152, in get_response
    extra={'status_code': 404, 'request': request},
Message: '\x1b[2m2017-05-08 18:34:53\x1b[0m [\x1b[33m\x1b[1mwarning  \x1b[0m] \x1b[1mNot Found: /favicon.ico\x1b[0m'
Arguments: ('/favicon.ico',)

I've tried to figure out what exactly goes on but lost my way in the debugger.

Of course, I can use structlog just for application logging, and keep the standard library loggers like they are. However, I want all logging unified, so my application's output would be uniform, ready for parsing.

I'd greatly appreciate a code snippet that shows how to integrate structlog with Django correctly.

Answer 1

It’s most likely this bug that will be fixed in structlog 17.2 that should be released soonish: https://github.com/hynek/structlog/pull/117 (feel free to comment or to try out if it fixes your problem).