How to write a language with Python-like indentation in syntax?

Question

I'm writing a tool with it's own built-in language similar to Python. I want to make indentation meaningful in the syntax (so that tabs and spaces at line beginning would represent nesting of commands).

What is the best way to do this?

I've written recursive-descent and finite automata parsers before.

Answer 1

The current CPython's parser seems to be generated using something called ASDL.

Regarding the indentation you're asking for, it's done using special lexer tokens called INDENT and DEDENT. To replicate that, just implement those tokens in your lexer (that is pretty easy if you use a stack to store the starting columns of previous indented lines), and then plug them into your grammar as usual (like any other keyword or operator token).