How to capture a value in a Python Closure

Question

How does one capture a value (as opposed to a reference) in a python closure?

What I have tried:

Here is a function which makes a list of parameter-less functions, each of which spits out a string:

def makeListOfFuncs( strings):
  list = []
  for str in strings:
    def echo():
      return str
    list.append( echo)
  return list

If closures in python worked like every other language, I would expect that a call like this:

for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
  print( animalFunc())

... would yield output like this:

bird
fish
zebra

But in python instead, we get:

zebra
zebra
zebra

Apparently what is happening is that the closure for the echo function is capturing the reference to str, as opposed to the value in the call frame at the time of closure construction.

How can I define makeListOfFuncs, so that I get 'bird', 'fish', 'zebra'?

Answer 1

Python closure is not weird if you know how closure internally works in python.

def makeListOfFuncs( strings):
  list = []
  for str in strings:
    def echo():
      return str
    list.append( echo)
  return list

You are returning a list of closures. variable "str" is shared between scopes, it is in two different scopes. When python sees it, it creates an intermediary object. this object contains a reference to another object which is "str". and for each closure, it is the same cell. You can test it:

closures_list= makeListOfFuncs( ['bird', 'fish', 'zebra'])
 # Those will return same cell address
 closures_list[0].__closure__
 closures_list[1].__closure__
 closures_list[2].__closure__

When you call makeListOfFuncs, it will run the for-loop and at the end of the loop, the intermediary object will point to "zebra". So when you call each closure print( animalFunc()) , it will visit the intermediary obj which will reference to "zebra".

You have to think about what happens when python creates a function and when it evaluates it.

def echo():
  return str

We need to somehow able to capture the value of "str" as the function being created. Because if you wait until function gets evaluated, then it is too late. Another solution would be defining a default value:

def makeListOfFuncs( strings):
    list = []
    for str in strings:
        def echo(y=str):
            return y
        list.append( echo)
    return list

for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
    print( animalFunc())

This solution works because default values get evaluated at creation time. So when echo is created, the default value of "str" will be used. the default value will not point to the cell.

in the first iteration, the default value will be "bird". Python will not see this as a free variable. in this case we are not even creating closure, we are creating function. in next iteration "fish" and in the last iteration "zebra" will be printed

Answer 2

I found the answer here on a blog post by Bob Kerner.

def makeListOfFuncs( strings):
  list = []
  for str in strings:
    def generateFunc( str):
      def echo():
        return str
      return echo
    list.append( generateFunc( str))
  return list

for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
  print( animalFunc())

Python closures are weird.