Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python AST with preserved comments

I can get AST without comments using

import ast module = ast.parse(open('/path/to/module.py').read()) 

Could you show an example of getting AST with preserved comments (and whitespace)?

like image 429
Andrei Avatar asked Sep 17 '11 18:09

Andrei


People also ask

What is ast Literal_eval in Python?

The ast. literal_eval method is one of the helper functions that helps traverse an abstract syntax tree. This function evaluates an expression node or a string consisting of a Python literal or container display.

Does Python use ast?

The ast module helps Python applications to process trees of the Python abstract syntax grammar. The abstract syntax itself might change with each Python release; this module helps to find out programmatically what the current grammar looks like. An abstract syntax tree can be generated by passing ast.

How do you define ast in Python?

For is a class defined in the ast module that expresses a for loop in Python in the form of an Abstract Syntax Tree. When the parse() method of ast is called on a Python source code that contains for loops, the ast. For class is invoked, which expresses the for statement to a node in an ast tree data structure.

Is an ast a binary tree?

Binary trees have many uses. In fact, compilers often use them to build what is known as an abstract syntax tree (or AST) - an intermediate representation of the code that is not yet fully compiled. The JavaScript parser uses an AST as well, though it's a general tree, not a binary tree.


2 Answers

The ast module doesn't include comments. The tokenize module can give you comments, but doesn't provide other program structure.

like image 128
Ned Batchelder Avatar answered Sep 29 '22 01:09

Ned Batchelder


An AST that keeps information about formating, comments etc. is called a Full Syntax Tree.

redbaron is able to do this. Install with pip install redbaron and try the following code.

import redbaron  with open("/path/to/module.py", "r") as source_code:     red = redbaron.RedBaron(source_code.read())  print (red.fst()) 
like image 25
azmeuk Avatar answered Sep 29 '22 01:09

azmeuk