Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: Lazy Function Evaluation in any() / all()

Logical operators in Python are lazy. With the following definition:

def func(s):
    print(s)
    return True

calling the or operator

>>> func('s') or func('t')
's'

only evaluates the first function call, because or recognizes that the expression evaluates to True, irregardless of the return value of the second function call. and does behave analogously.

However, when using any() (analogously: all()) in the following way:

>>> any([func('s'), func('t')])
's'
't'

all function calls are evaluated, because the inner list is constructed first, before any starts to iterate over the boolean values of its items. The same happens when we omit the list construction and just write

>>> any(func('s'), func('t'))
's'
't'

That way we lose the power of any being short-circuit, which means that it breaks as soon as the first element of the iterable is truish. If the function calls are expensive, evaluating all the functions up front is a big loss and is a waste of this ability of any. In some sense, one could call this a Python gotcha, because it might be unexpected for users trying to leverage this feature of any, and because any is often thought as being just another syntactic way of chaining a sequence of or statements. But any is just short-circuit, not lazy, and that is a difference here.

any is accepting an iterable. So, there should be a way of creating an iterator which does not evaluate its elements up front but pass them unevaluated to any and lets them evaluate inside of any only, in order to achieve a fully lazy evaluation.

So, the question is: How can we use any with truly lazy function evaluation? That means: How can we make an iterator of function calls which any can consume, without evaluating all the function calls in advance?

like image 650
jonathan.scholbach Avatar asked Sep 27 '20 16:09

jonathan.scholbach


People also ask

What is lazy evaluation in Python?

If you've never heard of Lazy Evaluation before, Lazy Evaluation is an evaluation strategy which delays the evaluation of an expression until its value is needed and which also avoids repeated evaluations (From Wikipedia). It's usually being considered as a strategy to optimize your code.

Does Python support lazy evaluation?

Yes, Python evaluates boolean conditions lazily. The docs say, The expression x and y first evaluates x; if x is false, its value is returned; otherwise, y is evaluated and the resulting value is returned.

What is the point of lazy evaluation?

The benefits of lazy evaluation include: The ability to define control flow (structures) as abstractions instead of primitives. The ability to define potentially infinite data structures. This allows for more straightforward implementation of some algorithms.

Does Python evaluate both sides of or?

In the last two examples, the left operand is false (an empty object). The Python or operator evaluates both operands and returns the object on the right, which may evaluate to either true or false.


1 Answers

We can use a generator expression, passing the functions and their arguments separately and evaluating only in the generator like so:

>>> any(func(arg) for arg in ('s', 't'))
's'

For different functions with different signatures, this could look like the following:

any(
    f(*args)
    for f, args in [(func1, ('s',)), (func2, (1, 't'))]
)

That way, any will stop calling the next() element in the generator as soon as one function call in the generator evaluates to True, and that means that the function evaluation is fully lazy.

Another neat way to postpone the function evaluation was mentioned by wjandrea in a comment: We can also to use lambda expressions, like so:

>>> any(f() for f in [lambda: func('s'), lambda: func('t')]
's'
like image 189
jonathan.scholbach Avatar answered Sep 18 '22 13:09

jonathan.scholbach