In Python, how can I get the global variables that are used in a function?

Question

I'm trying to collect info on crashes and I am having trouble figuring out how to get the globals that are being used in the crashed function.

import inspect

fun = 222
other = "junk"

def test():
    global fun
    harold = 888 + fun
    try:
        harold/0
    except:
        frames = inspect.trace()
        print "Local variables:"
        print frames[0][0].f_locals

        print "All global variables, not what I want!"
        print frames[0][0].f_globals

test()

test() only uses "fun" but f_globals gives all the available globals. Is there some way to get just the globals that are being used by this function?

Answer 1

Check this out

a = 10

def test():
    global a
    a = 12
    b = 12

print "co_argcount = ",test.__code__.co_argcount
print "co_cellvars = ",test.__code__.co_cellvars
print "co_code = ",test.__code__.co_code
print "co_consts = ",test.__code__.co_consts
print "co_filename = ",test.__code__.co_filename
print "co_firstlineno = ",test.__code__.co_firstlineno
print "co_flags = ",test.__code__.co_flags
print "co_freevars = ",test.__code__.co_freevars
print "co_lnotab = ",test.__code__.co_lnotab
print "co_name = ",test.__code__.co_name
print "co_names = ",test.__code__.co_names
print "co_nlocals = ",test.__code__.co_nlocals
print "co_stacksize = ",test.__code__.co_stacksize
print "co_varnames = ",test.__code__.co_varnames

Answer 2

I needed that also myself. This is my solution. The non-fast path covers most cases you are probably interested in.

def iterGlobalsUsedInFunc(f, fast=False, loadsOnly=True):
    if hasattr(f, "func_code"): code = f.func_code
    else: code = f
    if fast:
        # co_names is the list of all names which are used.
        # These are mostly the globals. These are also attrib names, so these are more...
        for name in code.co_names:
            yield name
    else:
        # Use the disassembly. Note that this will still not
        # find dynamic lookups to `globals()`
        # (which is anyway not possible to detect always).
        import dis
        ops = ["LOAD_GLOBAL"]
        if not loadsOnly:
            ops += ["STORE_GLOBAL", "DELETE_GLOBAL"]
        ops = map(dis.opmap.__getitem__, ops)
        i = 0
        while i < len(code.co_code):
            op = ord(code.co_code[i])
            i += 1
            if op >= dis.HAVE_ARGUMENT:
                oparg = ord(code.co_code[i]) + ord(code.co_code[i+1])*256
                i += 2
            else:
                oparg = None
            if op in ops:
                name = code.co_names[oparg]
                yield name

    # iterate through sub code objects
    import types
    for subcode in code.co_consts:
        if isinstance(subcode, types.CodeType):
            for g in iterGlobalsUsedInFunc(subcode, fast=fast, loadsOnly=loadsOnly):
                yield g

An updated version might be here.

---

My use case:

I have some module (songdb) which has some global database objects and I wanted to lazily load them once I called a function which uses the global database variable. I could have manually decorated such functions with a lazy loader or I could automatically detect which functions need it by my iterGlobalsUsedInFunc function.

This is basically the code (full code; was actually extended for classes now), where init automatically decorates such functions:

DBs = {
    "songDb": "songs.db",
    "songHashDb": "songHashs.db",
    "songSearchIndexDb": "songSearchIndex.db",
    }
for db in DBs.keys(): globals()[db] = None

def usedDbsInFunc(f):
    dbs = []
    for name in utils.iterGlobalsUsedInFunc(f, loadsOnly=True):
        if name in DBs:
            dbs += [name]
    return dbs

def init():
    import types
    for fname in globals().keys():
        f = globals()[fname]
        if not isinstance(f, types.FunctionType): continue
        dbs = usedDbsInFunc(f)
        if not dbs: continue
        globals()[fname] = lazyInitDb(*dbs)(f)

def initDb(db):
    if not globals()[db]:
        globals()[db] = DB(DBs[db])

def lazyInitDb(*dbs):
    def decorator(f):
        def decorated(*args, **kwargs):
            for db in dbs:
                initDb(db)
            return f(*args, **kwargs)
        return decorated
    return decorator

Another solution would have been to use an object proxy which lazily loads the database. I have used that elsewhere in this project, thus I have also implemented such object proxy; if you are interested, see here: utils.py:ObjectProxy.

Answer 3

A dirty way would be to use inspect.getsourcelines() and search for lines containing global <varname>. There are no good methods for this, at least not in inspect module.