Lexical Analysis of Python Programming Language

Question

Does anyone know where a FLEX or LEX specification file for Python exists? For example, this is a lex specification for the ANSI C programming language: http://www.quut.com/c/ANSI-C-grammar-l-1998.html

FYI, I am trying to write code highlighting into a Cocoa application. Regex won't do it because I also want grammar parsing to fold code and recognize blocks.

Answer 1

Lex is typically just used for tokenizing, not full parsing. Projects that use flex/lex for tokenizing typically use yacc/bison for the actual parsing.

You may want to take a look at ANTLR, a more "modern" alternative to lexx & yacc.

The ANTLR Project has a Github repo containing many ANTLR 4 grammars including at least one for Python 3.