Accessing Local Variables Inside a Python Generator

Question

How would you access a local variable defined inside a Python generator from outside the generator?

I have a case where my generator manipulates a local state, and for unittests I want to inspect this state to ensure it contains the correct values.

I can't store the state to an instance variable (e.g. self.state = blah), because I might be creating multiple generators from the same class instance, meaning the generators might overwrite each other's state. I also can't return the state in the yield expression, because the state name may change or vary because individual generator instances.

e.g. I want to do something like this (albeit this code doesn't work)

from random import random

class MyIter(object):
    def __iter__(self):
        context = {}
        for i in xrange(10):
            context[random()] = random()
            yield i

obj = MyIter()
i1 = iter(obj)
i2 = iter(obj)
while 1:
    try:
        i1.next()
        i2.next()
        print i1.context
        print i2.context
    except StopIteration:
        break

Is there anyway to access local variables by inspecting Python's execution stack?

Answer 1

Sorry to answer my own question, but after digging into the generator interface, I found the exact path I need to access the generator's local variables:

from random import random

class MyIter(object):
    def __iter__(self):
        context = {}
        for i in xrange(10):
            context[random()] = random()
            yield i

obj = MyIter()
i1 = iter(obj)
i2 = iter(obj)
while 1:
    try:
        i1.next()
        i2.next()
        print i1.gi_frame.f_locals['context']
        print i2.gi_frame.f_locals['context']
    except StopIteration:
        break