Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Lazy evaluation of strings in python logging: comparing `%` with `.format`

Is there any difference between these two calls:

import logging

logging.getLogger().debug('test: %i' % 42)

and

logging.getLogger().debug('test: {}'.format(42))

Where we assume 42 is replaced by some long computation (say 7.5 million years of computation) when it is cast to a string that produces a final answer of 42.

Is the first approach lazily evaluated in case logging is set to debug?

like image 545
David Parks Avatar asked Aug 24 '18 22:08

David Parks


People also ask

What is the difference between string and format?

There's no difference in terms of the output it produces. The result variable will have the same contents in each case. But String. format() is there so that you can do more powerful formatting, including specifying number of decimal places for floating point numbers, and the like.

Why not use f-strings for logging?

The problem with logging f-stringsIf it has the value %(foo)999999999s (where foo is present in some_dict ), logging gets stuck trying to add over a gigabyte of whitespace to a string. In other words: a Denial-of-Service attack.

What is lazy evaluation in python?

If you've never heard of Lazy Evaluation before, Lazy Evaluation is an evaluation strategy which delays the evaluation of an expression until its value is needed and which also avoids repeated evaluations (From Wikipedia). It's usually being considered as a strategy to optimize your code.

What is the string .format method used for in Python?

Python's str. format() method of the string class allows you to do variable substitutions and value formatting. This lets you concatenate elements together within a string through positional formatting.


2 Answers

Neither are lazy. Both strings are interpolated before sent to logger. Lazy evaluation in terms of python logging is done with separate arguments. The documentation https://docs.python.org/2/library/logging.html suggest the following for lazy evaluation of string interpolation;

logging.getLogger().debug('test: %i', 42)

TL;DR In this case it’s easier to consider the following. We sent a primitive type (string) but only one argument to the logger. Thus it can’t be lazy.

like image 72
Martin Avatar answered Sep 23 '22 14:09

Martin


I'd take a look in the references I posted in the comments for more detailed information on % and .format().

For the lazy evaluation question, the answer is no.

A simple test will do

def func1(x):
    time.sleep(5)
    return(x)

def func2(x):
    #time.sleep(5)
    return(x)

%timeit 'debug1: %s' % func1(3)
5 s ± 1.31 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
%timeit 'debug2: {}'.format(func1(3))
5 s ± 1.45 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit 'debug1: %s' % func2(3)
297 ns ± 11.2 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
%timeit 'debug2: {}'.format(func2(3))
404 ns ± 4.56 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In both .format (obviously) and % approaches,func() is calculated anyway.

like image 27
rafaelc Avatar answered Sep 25 '22 14:09

rafaelc