Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I can format exception stacktraces in Python logging?

The logs I am creating in Python are intended to be temporarily stored as files which will, in turn, be processed into a log database. They take a pipe-delineated format to dictate how the logs will be processed, but logging.exception() is breaking my standard by adding one too many fields and way too many newlines.

import logging
logging.basicConfig(filename='output.txt', 
                    format='%(asctime)s|%(levelname)s|%(message)s|', 
                    datefmt='%m/%d/%Y %I:%M:%S %p', 
                    level=logging.DEBUG)
logging.info('Sample message')

try:
    x = 1 / 0
except ZeroDivisionError as e:
    logging.exception('ZeroDivisionError: {0}'.format(e))

# output.txt
01/27/2015 02:09:01 PM|INFO|Sample message|
01/27/2015 02:09:01 PM|ERROR|ZeroDivisionError: integer division or modulo by zero|
Traceback (most recent call last):
  File "C:\Users\matr06586\Desktop\ETLstage\Python\blahblah.py", line 90, in <module>
    x = 1 / 0
ZeroDivisionError: integer division or modulo by zero

How can I best handle or format tracebacks with the whitespace and newlines? These messages are part and parcel in logging.exception(), but it feels odd to circumvent the function when I am attempting to document instances of exceptions. How do I record my tracebacks and format them too? Should the tracebacks be ignored?

Thank you for your time!

like image 689
twoxmachine Avatar asked Jan 27 '15 21:01

twoxmachine


People also ask

How do I log an exception logger in Python?

To log an exception in Python we can use logging module and through that we can log the error. Logging an exception in python with an error can be done in the logging. exception() method. This function logs a message with level ERROR on this logger.

Does logging error raise exception Python?

It depends on the situation, but logging and then raising an exception is generally considered an antipattern. It's redundant and clutters logs. Unless you're expecting something to catch that exception and suppress the message, don't log. Pythonic is not really a look as it is the formatting of code.

Does logger exception throw exception?

You can log the problem in catch block and throw a new unchecked exception (of course with passing the previous one inside). In such solution there is no overflow in the logs and exception still bubble up to the highest level.


2 Answers

You can define your own Formatter whose methods you can override to format exception information exactly how you want it. Here is a simplistic (but working) example:

import logging

class OneLineExceptionFormatter(logging.Formatter):
    def formatException(self, exc_info):
        result = super(OneLineExceptionFormatter, self).formatException(exc_info)
        return repr(result) # or format into one line however you want to

    def format(self, record):
        s = super(OneLineExceptionFormatter, self).format(record)
        if record.exc_text:
            s = s.replace('\n', '') + '|'
        return s

fh = logging.FileHandler('output.txt', 'w')
f = OneLineExceptionFormatter('%(asctime)s|%(levelname)s|%(message)s|', '%m/%d/%Y %I:%M:%S %p')
fh.setFormatter(f)
root = logging.getLogger()
root.setLevel(logging.DEBUG)
root.addHandler(fh)
logging.info('Sample message')

try:
    x = 1 / 0
except ZeroDivisionError as e:
    logging.exception('ZeroDivisionError: {0}'.format(e))

This produces just two lines:

01/28/2015 07:28:27 AM|INFO|Sample message|
01/28/2015 07:28:27 AM|ERROR|ZeroDivisionError: integer division or modulo by zero|'Traceback (most recent call last):\n  File "logtest2.py", line 23, in <module>\n    x = 1 / 0\nZeroDivisionError: integer division or modulo by zero'|

Of course, you can build on this example to do precisely what you want, e.g. via the traceback module.

like image 58
Vinay Sajip Avatar answered Sep 27 '22 20:09

Vinay Sajip


For my usecase Vinay Sajip's code did not work good enough (I worked with more complex message format), so I came up with this one (for me it's also cleaner):

class OneLineExceptionFormatter(logging.Formatter):
    def format(self, record):
        if record.exc_info:
            # Replace record.msg with the string representation of the message
            # use repr() to prevent printing it to multiple lines
            record.msg = repr(super().formatException(record.exc_info))
            record.exc_info = None
            record.exc_text = None
        result = super().format(record)
        return result

So this format() method can detect that an exception is going to be logged and can just convert it to its string representation and formatting of the log message happen simply for that plain message string. I tested it in python 3.

like image 34
Attila123 Avatar answered Sep 27 '22 21:09

Attila123