Why does Python read from the current directory when printing a traceback?

Question

$ echo "Your code is bad and you should feel bad" > "<stdin>"
$ python
Python 3.6.0 (default, Dec 28 2016, 19:53:26) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 2 + '2'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    Your code is bad and you should feel bad
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Why does Python confuse the string "<stdin>" with a file matching that filename? I didn't want Python trying to just read whatever files from my disk if it encountered an unhandled exception.

You can also get it with the "<string>" filename:

$ echo "pining for the fjords" > "<string>"
$ python -c 'wat'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    pining for the fjords
NameError: name 'wat' is not defined

Is there any way to prevent that behaviour, or is it hardcoded into the REPL?

Answer 1

Python doesn't keep track of what source code corresponds to any compiled bytecode. It might not even read that source code until it needs to print a traceback, for example if a module is loaded from a .pyc file.

When Python needs to print a traceback, that's when it tries to find source code corresponding to all the stack frames involved. The file name and line number you see in the stack trace are all Python has to go on. If it were using the traceback module, the code path would go through a section in linecache that excludes filenames starting and ending with < and >, but the default sys.excepthook doesn't go through that path.

The default sys.excepthook goes through the native call PyErr_Display, which eventually winds up using _Py_DisplaySourceLine to display individual source lines. _Py_DisplaySourceLine unconditionally tries to find the file in the current working directory (for some reason - misguided optimization?), then calls _Py_FindSourceFile to search sys.path for a file matching that name if the working directory didn't have it. Usually, it won't find a <stdin> or <string> file, and it'll just skip printing source code when it can't find a file, but if it finds one, it prints from that file.

I initially thought you could prevent this by running Python with the -I flag, putting it in isolated mode. One of the effects of isolated mode is to remove the script's directory from sys.path. Experiment proved that this didn't change things, which is when I realized _Py_DisplaySourceLine tries the working directory no matter what.

It would be fairly straightforward to fix this by excluding <> filenames in the native code path, like linecache does. The code that unconditionally searches the current directory for the file should also be changed.