python-3.xclosures

How to capture a value in a Python Closure


How does one capture a value (as opposed to a reference) in a python closure?

What I have tried:

Here is a function which makes a list of parameter-less functions, each of which spits out a string:

def makeListOfFuncs( strings):
  list = []
  for str in strings:
    def echo():
      return str
    list.append( echo)
  return list

If closures in python worked like every other language, I would expect that a call like this:

for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
  print( animalFunc())

... would yield output like this:

bird
fish
zebra

But in python instead, we get:

zebra
zebra
zebra

Apparently what is happening is that the closure for the echo function is capturing the reference to str, as opposed to the value in the call frame at the time of closure construction.

How can I define makeListOfFuncs, so that I get 'bird', 'fish', 'zebra'?


Solution

  • Python closure is not weird if you know how closure internally works in python.

    def makeListOfFuncs( strings):
      list = []
      for str in strings:
        def echo():
          return str
        list.append( echo)
      return list
    

    You are returning a list of closures. variable "str" is shared between scopes, it is in two different scopes. When python sees it, it creates an intermediary object. this object contains a reference to another object which is "str". and for each closure, it is the same cell. You can test it:

    closures_list= makeListOfFuncs( ['bird', 'fish', 'zebra'])
     # Those will return same cell address
     closures_list[0].__closure__
     closures_list[1].__closure__
     closures_list[2].__closure__
    

    enter image description here

    When you call makeListOfFuncs, it will run the for-loop and at the end of the loop, the intermediary object will point to "zebra". So when you call each closure print( animalFunc()) , it will visit the intermediary obj which will reference to "zebra".

    You have to think about what happens when python creates a function and when it evaluates it.

    def echo():
      return str
    

    We need to somehow able to capture the value of "str" as the function being created. Because if you wait until function gets evaluated, then it is too late. Another solution would be defining a default value:

    def makeListOfFuncs( strings):
        list = []
        for str in strings:
            def echo(y=str):
                return y
            list.append( echo)
        return list
    
    for animalFunc in makeListOfFuncs( ['bird', 'fish', 'zebra']):
        print( animalFunc())
    

    This solution works because default values get evaluated at creation time. So when echo is created, the default value of "str" will be used. the default value will not point to the cell.

    in the first iteration, the default value will be "bird". Python will not see this as a free variable. in this case we are not even creating closure, we are creating function. in next iteration "fish" and in the last iteration "zebra" will be printed