Why do indented explicit line continuations not allow comments in Python?

Question

I'm writing a Python parser to learn Flex and Bison, and I'm trying to find out why only the first of these programs is valid Python.

a.py:

\
# This is valid Python

produces no error.

b.py:

    \
# This is not valid Python

produces this error:

  File "b.py", line 1
    \
    ^
IndentationError: unexpected indent

and c.py:

if True:
    pass
    \
    # This is not valid Python

produces this error:

  File "c.py", line 4
    # This is not valid Python
                             ^
SyntaxError: invalid syntax

I'm using Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56) [GCC 4.4.3] on linux2 (Ubuntu 10.04); However, testing on ideone.com suggests the behavior is the same on Python 3.

Answer 1

It's an implementation detail.

Here's how a few different implementations respond to your code:

                 a.py  b.py  c.py    
                 ----  ----  ----
CPython 2.6.5     ok    bad   bad
CPython 3.?       ok    bad   bad
Jython 2.2.1      ok    ok    bad
Jython 2.5.2      bad   bad   bad
IronPython 2.7.1  ok    bad   ok

My reading of the Exlplicit Line Joining section of the Python Language Reference is that all three examples could be treated as valid:

Two or more physical lines may be joined into logical lines using backslash characters (\), as follows: when a physical line ends in a backslash that is not part of a string literal or comment, it is joined with the following forming a single logical line, deleting the backslash and the following end-of-line character.

If CPython were changed to accept all three examples as valid I doubt it would be noticed by its users, change the character of the language, or break any code.

Answer 2

Steven's quote is relevant, but it still doesn't directly explain this situation.

I think the key insight is that a line continuation character makes Python treat a line as not just whitespace.

• a.py: It seems like it's treating the first line as whitespace. It's not; once the line continuation character is reached, it and the newline are removed, and so since there is nothing else on that line it doesn't exist for the purposes of parsing -- you just have a single line with a comment. Note: Jython 2.5.2 treats this basically as expected; valid Python code is expected on a later line.

• b.py: Never makes it to the comment, as soon as the line continuation character is reached, and the line is not just whitespace any more, the indentation becomes an error.

• c.py: The comment is again irrelevant, you'll get the same error with any amount of whitespace and / or a comment on the next line. You need to have actual Python code on the line following the line continuation character.