How to find list comprehension in python code

Question

I want to find a list comprehension in python source code, for that I tried to use Pygments, but it didn't find the way to do that.

To be more specific, I want to do a function that recognize all the posible list comprehension. For example:

[x**2 for x in range(5)]

[x for x in vec if x >= 0]

[num for elem in vec for num in elem]

[str(round(pi, i)) for i in range(1, 6)]

This examples are obtained from https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions

It is also valid a solution with regular expression.

Thank you

Answer 1

You can use the ast library to parse Python code into a syntax tree and then walk the parsed tree to look for ListComp expressions.

Here's a simple example which prints the line numbers at which list comprehensions were found in the Python code passed via stdin:

import ast
import sys

prog = ast.parse(sys.stdin.read())
listComps = (node for node in ast.walk(prog) if type(node) is ast.ListComp)
for comp in listComps:
    print "List comprehension at line %d" % comp.lineno

Answer 2

You may use the ast built-in module.

import ast

my_code = """
print("Hello")
y = [x ** 2 for x in xrange(30)]
"""

module = ast.parse(my_code)
for node in ast.walk(module):
    if type(node) == ast.ListComp:
        print(node.lineno)  # 3
        print(node.col_offset)  # 5
        print(node.elt)  # <_ast.BinOp object at 0x0000000002326EF0>