Python equivalent of Scala's lazy val

Question

I'm currently trying to port some Scala code to a Python project and I came across the following bit of Scala code:

  lazy val numNonZero = weights.filter { case (k,w) => w > 0 }.keys

weights is a really long list of tuples of items and their associated probability weighting. Elements are frequently added and removed from this list but checking how many elements have a non-zero probability is relatively rare. There are a few other rare-but-expensive operations like this in the code I'm porting that seem to benefit greatly from usage of lazy val. What is the most idiomatic Python way to do something similar to Scala's lazy val?

Answer 1

In Scala, lazy val is a final variable that is evaluated once at the time it is first accessed, rather than at the time it is declared. It is essentially a memoized function with no arguments. Here's one way you can implement a memoization decorator in Python:

from functools import wraps

def memoize(f):
    @wraps(f)
    def memoized(*args, **kwargs):
        key = (args, tuple(sorted(kwargs.items()))) # make args hashable
        result = memoized._cache.get(key, None)
        if result is None:
            result = f(*args, **kwargs)
            memoized._cache[key] = result
        return result
    memoized._cache = {}
    return memoized

---

Here's how it can be used. With property you can even drop the empty parentheses, just like Scala:

>>> class Foo:
...     @property
...     @memoize
...     def my_lazy_val(self):
...         print "calculating"
...         return "some expensive value"

>>> a = Foo()
>>> a.my_lazy_val
calculating
'some expensive value'

>>> a.my_lazy_val
'some expensive value'

Answer 2

You can use @functools.lru_cache(maxsize=None) on a nullary function to emulate a lazy val.

Python 3.6.5 (default, Mar 30 2018, 06:41:53) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import functools
>>> import random
>>> def foo():
...     @functools.lru_cache(maxsize=None)
...     def bar():
...         return random.random()
...     return bar
... 
>>> f1 = foo()
>>> f2 = foo()
>>> f1()
0.11043217592970156
>>> f1()
0.11043217592970156
>>> f2()
0.3545457696543922
>>> f2()
0.3545457696543922

Answer 3

Essentially, you want to change how attribute access works for numNonZero. Python does that with a descriptor. In particular, take a look at their application to Properties.

With that, you can defer calculation until the attribute is accessed, caching it for later use.

Answer 4

Generator expression

>>> weights = [(1,2), (2,0), (3, 1)]
>>> numNonZero = (k for k, w in weights if w > 0)
>>> next(numNonZero)
1
>>> next(numNonZero)
3
>>> next(numNonZero)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> next(numNonZero, -1)
-1

---

>>> numNonZero = (k for k, w in weights if w > 0)
>>> for k in numNonZero:
...     print(k)
... 
1
3

Python tutorial: Generator expressions

Answer 5

A simpler variant of @sam-thomson's approach (inspired by his approach):

In [1]:     class Foo:
   ...:         def __init__(self):
   ...:             self.cached_lazy_val=None
   ...:         @property
   ...:         def my_lazy_val(self):
   ...:             if not self.cached_lazy_val:
   ...:                 print("calculating")
   ...:                 self.cached_lazy_val='some expensive value'
   ...:             return self.cached_lazy_val
   ...:

In [2]: f=Foo()

In [3]: f.my_lazy_val
calculating
Out[3]: 'some expensive value'

In [4]: f.my_lazy_val
Out[4]: 'some expensive value'