Python Lambdas and Variable Bindings

Question

I've been working on a basic testing framework for an automated build. The piece of code below represents a simple test of communication between two machines using different programs. Before I actually do any tests, I want to completely define them - so this test below is not actually run until after all the tests have been declared. This piece of code is simply a declaration of a test.

remoteTests = []
for client in clients:
    t = Test(
        name = 'Test ' + str(host) + ' => ' + str(client),
        cmds = [
            host.start(CMD1),
            client.start(CMD2),

            host.wait(5),

            host.stop(CMD1),
            client.stop(CMD2),
        ],
        passIf = lambda : client.returncode(CMD2) == 0
    )
remoteTests.append(t)

Anyhow, after the test is run, it runs the function defined by 'passIf'. Since I want to run this test for multiple clients, I'm iterating them and defining a test for each - no big deal. However, after running the test on the first client, the 'passIf' evaluates on the last one in the clients list, not the 'client' at the time of the lambda declaration.

My question, then: when does python bind variable references in lambdas? I figured if using a variable from outside the lambda was not legal, the interpretor would have no idea what I was talking about. Instead, it silently bound to the instance of the last 'client'.

Also, is there a way to force the resolution the way I intended it?

Answer 1

The client variable is defined in the outer scope, so by the time the lambda is run it will always be set to the last client in the list.

To get the intended result, you can give the lambda an argument with a default value:

passIf = lambda client=client: client.returncode(CMD2) == 0

Since the default value is evaluated at the time the lambda is defined, its value will remain correct.

Another way is to create the lambda inside a function:

def createLambda(client):
    return lambda: client.returncode(CMD2) == 0
#...
passIf = createLambda(client)

Here the lambda refers to the client variable in the createLambda function, which has the correct value.

Answer 2

What happens is that your passIf argument, the lambda, refers to the variable client from the enclosing scope. It doesn't refer to the object the variable client refers to when it is created, but the variable itself. If you call these passIf after the loop has ended, that means they all refer to the last value in the loop. (In closure terminology, Python's closures are late-binding, not early-binding.)

Fortunately it's fairly easy to make a late-binding closure into an early-binding closure. You can do it by simply giving the lambda an argument with as default the value you want to bind:

passIf = lambda client=client: client.returncode(CMD2) == 0

That does mean the function gets that extra argument, and might mess things up if it gets called with an argument by accident -- or when you want the function to take arbitrary arguments. So another technique is to do it like this:

# Before your loop:
def make_passIf(client):
    return lambda: client.returncode(CMD2) == 0

# In the loop
t = Test(
    ...
    passIf = make_passIf(client)
)