How can I strip Python logging calls without commenting them out?

Question

Today I was thinking about a Python project I wrote about a year back where I used logging pretty extensively. I remember having to comment out a lot of logging calls in inner-loop-like scenarios (the 90% code) because of the overhead (hotshot indicated it was one of my biggest bottlenecks).

I wonder now if there's some canonical way to programmatically strip out logging calls in Python applications without commenting and uncommenting all the time. I'd think you could use inspection/recompilation or bytecode manipulation to do something like this and target only the code objects that are causing bottlenecks. This way, you could add a manipulator as a post-compilation step and use a centralized configuration file, like so:

[Leave ERROR and above] my_module.SomeClass.method_with_lots_of_warn_calls  [Leave WARN and above] my_module.SomeOtherClass.method_with_lots_of_info_calls  [Leave INFO and above] my_module.SomeWeirdClass.method_with_lots_of_debug_calls 

Of course, you'd want to use it sparingly and probably with per-function granularity -- only for code objects that have shown logging to be a bottleneck. Anybody know of anything like this?

Note: There are a few things that make this more difficult to do in a performant manner because of dynamic typing and late binding. For example, any calls to a method named debug may have to be wrapped with an if not isinstance(log, Logger). In any case, I'm assuming all of the minor details can be overcome, either by a gentleman's agreement or some run-time checking. :-)

Answer 1

What about using logging.disable?

I've also found I had to use logging.isEnabledFor if the logging message is expensive to create.

Answer 2

Use pypreprocessor

Which can also be found on PYPI (Python Package Index) and be fetched using pip.

Here's a basic usage example:

from pypreprocessor import pypreprocessor  pypreprocessor.parse()  #define nologging  #ifdef nologging ...logging code you'd usually comment out manually... #endif 

Essentially, the preprocessor comments out code the way you were doing it manually before. It just does it on the fly conditionally depending on what you define.

You can also remove all of the preprocessor directives and commented out code from the postprocessed code by adding 'pypreprocessor.removeMeta = True' between the import and parse() statements.

The bytecode output (.pyc) file will contain the optimized output.

SideNote: pypreprocessor is compatible with python2x and python3k.

Disclaimer: I'm the author of pypreprocessor.