How to parse Python 2.x with Python 3.x ast module?

Question

I recently wrote a Sublime Text 3 plugin that parses Python code and issues some stats from it.

Nothing too complex, except I noticed Sublime Text 3 uses Python 3.x and apparently the ast module expects to see Python 3.x code as well. I looked at the documentation for the ast module but couldn't find anything that allows me to specify the version.

Obviously this causes issues when parsing perfectly valid Python 2.7 code.

Is there a way to specify or somehow detect/guess the Python version ? Or to allow the parser to be more flexible ?

Answer 1

No, the ast module can only handle Python code for the version of Python it is shipped with. That's because under the hood, the exact same parser is used to compile your Python code into byte code; all the ast module does is give you the intermediary AST tree. This parser has no 'backwards compatible' mode.

You'll need to implement your own parser if you want to parse different source code from Python versions. This certainly appears the path that Jedi (a Python autocompletion library) took.

Looking at the SublimeCodeIntel packaging of CodeIntel2, I see that that package bases its Python and Python 3 parsers on SilverCity, which has Python bindings. Perhaps that project would be a suitable starting point for your own.