IndentationError when pasting code in Python 3 interpreter mode

Question

When I was running the following code, which has blank lines (no space) inside functions, I got different behavior from Python 3.6.5 when running the code line by line in interpreter mode and python3 split.py:

# File split.py

def find(s, start, predictor):
    for i in range(start, len(s)):
        if predictor(s[i]):
            return i
    return -1

def find_all(s, sep=" \t\n"):
    beg_of_nonsep = 0
    while beg_of_nonsep < len(s):
        beg_of_nonsep = find(s, beg_of_nonsep, lambda ch, sep_chs=sep: ch not in sep_chs)
        if beg_of_nonsep == -1:
            break

        end_of_nonsep = find(s, beg_of_nonsep + 1, lambda ch, sep_chs=sep: ch in sep_chs)
        if end_of_nonsep == -1:
            end_of_nonsep = len(s)

        yield (beg_of_nonsep, end_of_nonsep)

        beg_of_nonsep = end_of_nonsep + 1

split = lambda s: [s[beg: end] for (beg, end) in find_all(s)]

print(split(""))
print(split("     \t\n"))
print(split("     \tssss\n"))

When running the code line by line in interpreter mode, python3 gave me nasty errors:

Type "help", "copyright", "credits" or "license" for more information.
>>> def find(s, start, predictor):
...     for i in range(start, len(s)):
...         if predictor(s[i]):
...             return i
...     return -1
...
>>> def find_all(s, sep=" \t\n"):
...     beg_of_nonsep = 0
...     while beg_of_nonsep < len(s):
...         beg_of_nonsep = find(s, beg_of_nonsep, lambda ch, sep_chs=sep: ch not in sep_chs)
...         if beg_of_nonsep == -1:
...             break
...
>>>         end_of_nonsep = find(s, beg_of_nonsep + 1, lambda ch, sep_chs=sep: ch in sep_chs)
  File "<stdin>", line 1
    end_of_nonsep = find(s, beg_of_nonsep + 1, lambda ch, sep_chs=sep: ch in sep_chs)
    ^
IndentationError: unexpected indent
>>>         if end_of_nonsep == -1:
  File "<stdin>", line 1
    if end_of_nonsep == -1:
    ^
IndentationError: unexpected indent
>>>             end_of_nonsep = len(s)
  File "<stdin>", line 1
    end_of_nonsep = len(s)
    ^
IndentationError: unexpected indent
>>>
>>>         yield (beg_of_nonsep, end_of_nonsep)
  File "<stdin>", line 1
    yield (beg_of_nonsep, end_of_nonsep)
    ^
IndentationError: unexpected indent
>>>
>>>         beg_of_nonsep = end_of_nonsep + 1
  File "<stdin>", line 1
    beg_of_nonsep = end_of_nonsep + 1
    ^
IndentationError: unexpected indent
>>>
>>> split = lambda s: [s[beg: end] for (beg, end) in find_all(s)]
>>>
>>> print(split(""))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
TypeError: 'NoneType' object is not iterable
>>> print(split("     \t\n"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
TypeError: 'NoneType' object is not iterable
>>> print(split("     \tssss\n"))

And the last print here never quit until I used ctrlc to stop it.

Thus, I thought there were many bugs with my code.

However, when I ran the code with python3 split.py, none of this happened:

[]
[]
['ssss']

This is rather confusing to me.

To be clear, I was actually using vimcmdline on Debian 9.8 with vim 8.1.

I am also using pylint through pymode, which split any trailing space, of which it considered superfluous.

This problem happened when I tried to use its builtin keybinding to send split.py to the python3 in the tmux split it opened.

I have already filled an issue, but I can't help wonder why python3 behaves like this.

Answer 1

This behaviour doesn't look surprising to me.

Python uses indentation to determine the beginning and end of a code block. Logically indentation ends on the first line that isn't indented. When running scripts blank lines are ignored. And when running scripts this means that indentation ends either with an un-indented line or the end of file.

But this behaviour cannot work in a command line mode because there is no end of file. Consider the following script file:

from somewhere import bar, do_something

for foo in bar:
    do_something(foo)

In a script, the end of the file indicates that it should now run the for loop. It knows there is no more to execute. But in command line mode, the command line is still open, you can still write more. It has no idea whether or not your next line of code is going to be inside or outside the for loop. But the command line cannot just sit and wait for your next line of code... you want it to execute ... now!

Therefore the command line operates with one specific difference. A blank line will also end a code block. So this is fine:

from somewhere import bar, do_something, do_something_else

for foo in bar:
    do_something(foo)
    do_something_else(foo)

But this is an error:

from somewhere import bar, do_something, do_something_else

for foo in bar:
    do_something(foo)

    do_something_else(foo)

Because you have already ended the for loop with a blank line and cannot add to it.