Closures: What is a good use case example? Why not a functor? And is it worth the negatives?

Question

I recently dove into Python. Previous, I had programmed mostly numerical and data analysis code in C++ and Matlab. I saw a lot of discussions about Python and Ruby and closures. Almost all examples looked like this:

>>> def makeAdder(y):
...  def myAdder(x):
...   return x + y
...  return myAdder
... 
>>> f = makeAdder(10)
>>> f(5)
15

I understand that this can be useful in some sense. However, realistically, the behavior in situations like this ('read only' situations as it were) can easily be emulated by an object (a functor):

>>> class MyAdder(object):
...  def __init__(self,y):
...   self.y = y
...  def __call__(self,x):
...   return self.y + x
... 
>>> f = MyAdder(5)
>>> f(10)
15

The object doesn't take up substantially more space to code, and it is far more versatile. It's also much easier to track and debug subsequent code.

In this case, we only read from the nonlocal variable. But we can also write to it: in Ruby, naturally, in Python by using the nonlocal keyword. The object supports that as well of course. But with the object, you have the data bundled together so you know exactly what's going on. The closure can potentially be carrying around variables in a totally non-transparent way and this can lead to code that's amazingly hard to debug. Here's a really bizarre example:

irb(main):001:0> def new_counter
irb(main):002:1> x = 0
irb(main):003:1> lambda { x +=1 }
irb(main):004:1> end
=> nil
irb(main):005:0> counter_a = new_counter
=> #<Proc:0x00007f85c6421cd0@(irb):3>
irb(main):006:0> counter_a.call
=> 1
irb(main):007:0> counter_a.call
=> 2

At least to me, this behavior is unintuitive. It also has the potential to cause memory leaks. This gives you a huge amount of rope to hang yourself with. Again, this is especially true in Ruby where you do not need to enable this explicitly (unlike Python), and because in Ruby one has blocks all over their main code, which have access to everything. If an outside variable gets changed as a result of being in a closure, if you pass that closure around you can have a variable changed indefinitely far and out of scope from the place where it lives. Contrast to an object that always safely carries its data with it.

Why do you hear a lot of talk about how good closures are, and how they should be potentially included in Java, how it sucked when they weren't fully in Python, etc. ? Why not use a functor? Or refactor the code to avoid, given how incredibly dangerous they can be? Just to clarify, I'm not one of those foaming at the mouth OO types. Have I underestimated their use, overstated their danger, or both?

Edit: maybe I should distinguish between three things: closures that only read once (which is what my example shows, and almost everyone discusses), closures that read in general, and closures that write. If you define a function inside another function using a variable local to the outer function, there's almost no chance this will come back to haunt you. The variable in that space is not accessible in any way I can think of, so you can't change it. This is pretty safe, and a convenient (possibly more than functors) way to generate functions. On the other hand, if you create a closure inside a class method or inside the main thread, it will read in variables each time that are called that can be accessed from other places. So it can change. I think this is dangerous because the closed over variable does not appear in the function header. You could have say a long closure on page 1 of your code that closes over a main thread variable x, and then modify x for unrelated reasons. Then re-use the closure and get bizarre behavior you don't understand, which may be hard to debug. If you actually write to enclosed variables, then as my example with Ruby shows you really have the potential to make a mess and cause unexpected behavior.

Edit2: I gave an example of bizarre behavior from closures for the third usage, writing to non local variables. Here's an example of bizarre (not as bad) behavior from the second usage (defining closures in scopes where their closed-over variables can be modified):

>>> fs = [(lambda n: i + n) for i in range(10)]
>>> fs[4](5)
14

Answer 1

Readability. Your Python example shows how much more obvious and easy to read the closure version is compared to the functor.

We also neatly avoid making a class that does nothing but act like a function - this smells of redundancy.

If nothing else, when we are doing something, it makes sense to describe it as an action, not an object.

As another note, an example of where these structures are used a lot in Python, to great effect, is decorators. Most function decorators do something like this, and they are a really useful feature.

Edit: As a note on state, remember that functions are not special in Python, they are still objects:

>>> def makeAdder(y):
...     def myAdder(x):
...         return x + myAdder.y
...     myAdder.y = y
...     return myAdder
... 
>>> f = makeAdder(10)
>>> f(5)
15
>>> f.y
10