How do you turn an unquoted Python function/lambda into AST? 2.6

Question

This seems like it should be easy, but I can't find the answer anywhere - nor able to derive one myself. How do you turn an unquoted python function/lambda into an AST?

Here is what I'd like to be able to do.

import ast
class Walker(ast.NodeVisitor):
    pass
    # ...

# note, this doesnt work as ast.parse wants a string
tree = ast.parse(lambda x,y: x+y)

Walker().visit(tree)

Answer 1

In general, you can't. For example, 2 + 2 is an expression -- but if you pass it to any function or method, the argument being passed is just the number 4, no way to recover what expression it was computed from. Function source code can sometimes be recovered (though not for a lambda), but "an unquoted Python expression" gets evaluated so what you get is just the object that's the expression's value.

What problem are you trying to solve? There may be other, viable approaches.

Edit: tx to the OP for clarifying. There's no way to do it for lambda or some other corner cases, but as I mention function source code can sometimes be recovered...:

import ast
import inspect

def f():
  return 23

tree = ast.parse(inspect.getsource(f))

print ast.dump(tree)

inspect.getsource raises IOError if it can't get the source code for whatever object you're passing it. I suggest you wrap the parsing and getsource call into an auxiliary function that can accept a string (and just parses it) OR a function (and tries getsource on it, possibly giving better errors in the IOError case).

Answer 2

If you only get access to the function/lambda you only have the compiled python bytecode. The exact Python AST can't be reconstructed from the bytecode because there is information loss in the compilation process. But you can analyze the bytecode and create AST's for that. There is one such analyzer in GeniuSQL. I also have a small proof of concept that analyzes bytecode and creates SQLAlchemy clauseelements from this.

The process I used for analyzing is the following:

1. Split the code into a list of opcodes with potential arguments.
2. Find the basic blocks in the code by going through the opcodes and for every jump create a basic block boundary after the jump and before the jump target
3. Create a control flow graph from the basic blocks.
4. Go through all the basic blocks with abstract interpretation tracking stack and variable assignments in SSA form.
5. To create the output expression just get the calculated SSA return value.

I have pasted my proof of concept and example code using it. This is non-clean quickly hacked together code, but you're free to build on it if you like. Leave a note if you decide to make something useful from it.

Answer 3

The Meta library allows you to recover the source in many cases, with some exceptions such as comprehensions and lambdas.

import meta, ast
source = '''
a = 1
b = 2
c = (a ** b)
'''

mod = ast.parse(source, '<nofile>', 'exec')
code = compile(mod, '<nofile>', 'exec')

mod2 = meta.decompile(code)
source2 = meta.dump_python_source(mod2)

assert source == source2