Django Error Logging: Adding request header, body and user information

Question

Looking for a way to add header, body and a user's email address in my error log along with the stack trace of the exception in my views.py

After scouring the web for hours, many suggested to write my own middleware and some suggested to log that sort of information into a separate log. However, knowing where your code went wrong solves one part of the problem, identifying which poor soul it affected and what request data was sent during that exception goes a long a way in rectifying the issue. Having that information in the same log file just makes sense to me.

Currently in my views.py, I have this simple setup:

from django.db.models import Min, Max, Q, F, Count, Sum
from django.db import connection
from django.conf import settings
from django.http import HttpResponse, HttpResponseRedirect
from myapp.models import *
import logging

logging.basicConfig(filename="errors.log",
                    level=logging.ERROR,
                    format='%(asctime)s: %(message)s')


def random_view(request):
    if request.user.is_authenticated() and request.user.is_active:
         # generic view code goes here.
    else:
        return HttpResponse(status=401)

This setup worked well for a while. Every time there was an exception, it would log out the time, the exception error message and the stack trace.

How can I also add in request.META, request.user.id and request.body along with stack trace?

Any suggestions would help. A worked out answer, even better!

Thank you

Answer 1

I think a complete solution to the logging problem you have is to implement a middleware. The middleware would be able to work with any kind of view implementation you have, irrespective of whether it is a class based view, function based view or APIView from DRF.

You can define a middleware for full logging. Make sure that you place the middleware appropriately after the authentication middleware -

MIDDLEWARE = [
    ...,
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    ...,
    'path.to.your.middleware.LogMiddleware'
]

In the log middleware, you would have access to the request and response. You can store request, the user (if authenticated) and all the META properties coming from the request via a logger, or you can even store it in a database if you want. Although, beware that storing in database comes at a cost. You can learn how to write a middleware by going through the Django middleware documentation.

import traceback

class LogMiddleware():
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        try:
            return self.get_response(request)
        except:
            if request.user.is_authenticated():
                # Log the user
            path = request.get_full_path() # Get the URL Path
            tb = traceback.format_exc() # Get the traceback
            meta = request.META # Get request meta information
            # Log everything
            raise  # Raise exception again after catching

You can read about all the meta attributes present from the django documentation of HttpRequest. Let me know if you need any clarification on this.

Answer 2

I would use a decorator here. Cut straight to the code...

import logging
from functools import wraps
from django.http import HttpResponse, HttpRequest

logging.basicConfig(filename="errors.log",
                    level=logging.ERROR,
                    format='%(asctime)s: %(message)s')
log = logging.getLogger(__name__)

def log_exceptions(wrapped):
    @wraps(wrapped)
    def wrapper(*args, **kwargs):
        try:
            return wrapped(*args, **kwargs)
        except:
            # log and re-raise
            request = args[0] if len(args) > 0 and isinstance(args[0], HttpRequest) else None
            msg = ("\nuser.id/email: {}/{}\nMETA: {}...\nbody: {}"
                   .format(request.user.id,
                           getattr(request.user, 'email','?'),
                           str(request.META)[:80],
                           request.body)
                   if request
                   else "not a HttpRequest")
            log.exception(msg)
            raise
    return wrapper

@log_exceptions
def random_view(request):
    raise ValueError("simulate a crash")
    if request.user.is_authenticated() and request.user.is_active:
        return HttpResponse('hi')
         # generic view code goes here.
    else:
        return HttpResponse(status=401)

and errors.log should capture something like

2017-06-27 20:48:09,282: 
user.id/email: 1/[email protected]
META: {'SESSION_MANAGER': 'local/acb:@/tmp/.ICE-unix/4255,unix/acb:/tmp/.ICE-unix/4255...
body: b''
Traceback (most recent call last):
  File "/home/rod/pyves/rangetest/rangetest/data/views.py", line 14, in wrapper
    return wrapped(*args, **kwargs)
  File "/home/rod/pyves/rangetest/rangetest/data/views.py", line 31, in random_view
    raise ValueError("simulate a crash")
ValueError: simulate a crash

Note, you'll also probably see the Django crash logging in your errors.log as well. You might split the logs to separate files using Django's well documented, but nonetheless complex logging config