Python 2 newline tokens in tokenize module

Question

I am using the tokenize module in Python and wonder why there are 2 different newline tokens:

NEWLINE = 4
NL = 54

Any examples of code that would produce both tokens would be appreciated.

Answer 1

According to python documentation:

tokenize.NL
Token value used to indicate a non-terminating newline. The NEWLINE token indicates the end of a logical line of Python code; NL tokens are generated when a logical line of code is continued over multiple physical lines.

More here: https://docs.python.org/2/library/tokenize.html