Python Logging vs performance

Question

I'm using python Logger in one of my programs.

The program is a solver for an np-hard problem and therefore uses deep iterations that run several times.

My question is if the Logger can be an issue in the performance of my program and if there are better ways to log information maintaining performance.

Answer 1

Depending on your Logger configuration and the amount of logs your program produces, yes, logging can be a performance bottleneck because of the blocking Logger operation. For example when directly logging to an NFS file from a NFS server with slow response time. One possible approach to improve performance in such case would be switching to use of a logserver able to buffer and possibly batch logging operations - the blocking would be limited to the communication with the logserver, not to the (slow) logfile access, which is often better from the performance prospective.