Can Python recognize changes to a file that it is running interactively?

Question

I was doing some troubleshooting and I was curious if it is possible to run a Python script interactively, change a function defined in the script, save the file, then have the interactive shell recognize the changes. Here is an example of what I am doing currently:

my_script.py:

def dummy_func():
    print('Something')
def main():
    dummy_func()
if __name__ == '__main__':
    main()

I go to my terminal and run:

>python -i my_script.py
Something
>>>

If I go back to my_script.py in my editor and make the following change:

def dummy_func():
    print('Something else')

Then go back to the terminal (which is still open) and re-run the updated function:

>>>dummy_func()
Something
>>>

Is it possible to do something to instead get the following behavior?:

>>>dummy_func()
Something else
>>>

I know it is possible to reload modules using importlib and reload but as far as I can tell that does not apply here since I am not importing anything.

I think this may be distinct from How do I unload (reload) a Python module?. I am asking if there is a way to reload the current file you are running interactively through the python shell, while that question is asking about reloading a module you have imported into another python script.

Answer 1

From what I can find, the short answer is:
No, normally the Python interpreter does not recognize changes to a file once that file has been parsed, analyzed, and fed into the interpreter.

What you should do instead apparently is use your .py file as a module, import that as a module into another .py file, then run that new file. This allows your first file to be reloaded through the interactive interpreter. Here's an example:

from importlib import reload  # Python 3.4+ only.
import foo

while True:
    # Do some things.
    if is_changed(foo):
        foo = reload(foo)

I am still a little fuzzy on the details, but maybe someone can help fill those in. As far as I can tell from the sources I linked below, the interpreter basically takes some steps to load your program from the saved python file into memory (glossing over a lot of details). Once this process has been performed, the interpreter does not perform it again unless you explicitly ask it to do so, for example by using the importlib's reload() function to again perform the process.

Sources:

How do I unload (reload) a Python module? (quoted above)

A Python Interpreter Written in Python:
This link has a lot more information about how the interpreter works, and I found this section particularly helpful:

Real Python Bytecode
At this point, we'll abandon our toy instruction sets and switch to real Python bytecode. The structure of bytecode is similar to our toy interpreter's verbose instruction sets, except that it uses one byte instead of a long name to identify each instruction. To understand this structure, we'll walk through the bytecode of a short function. Consider the example below:

>>> def cond():  
...     x = 3  
...     if x < 5:  
...         return 'yes'  
...     else:  
...         return 'no'  
...  

Python exposes a boatload of its internals at run time, and we can access them right from the REPL. For the function object cond, cond.code is the code object associated it, and cond.code.co_code is the bytecode. There's almost never a good reason to use these attributes directly when you're writing Python code, but they do allow us to get up to all sorts of mischief—and to look at the internals in order to understand them.

>>> cond.__code__.co_code  # the bytecode as raw bytes  
 b'd\x01\x00}\x00\x00|\x00\x00d\x02\x00k\x00\x00r\x16\x00d\x03\x00Sd\x04\x00Sd\x00\x00S'
>>> list(cond.__code__.co_code)  # the bytecode as numbers  
[100, 1, 0, 125, 0, 0, 124, 0, 0, 100, 2, 0, 107, 0, 0, 114, 22, 0, 100, 3, 0, 83,
100, 4, 0, 83, 100, 0, 0, 83]  

When we just print the bytecode, it looks unintelligible—all we can tell is that it's a series of bytes. Luckily, there's a powerful tool we can use to understand it: the dis module in the Python standard library.

dis is a bytecode disassembler. A disassembler takes low-level code that is written for machines, like assembly code or bytecode, and prints it in a human-readable way. When we run dis.dis, it outputs an explanation of the bytecode it has passed.

>>> dis.dis(cond)   
  2           0 LOAD_CONST               1 (3)
              3 STORE_FAST               0 (x)
  3           6 LOAD_FAST                0 (x)
              9 LOAD_CONST               2 (5)
             12 COMPARE_OP               0 (<)
             15 POP_JUMP_IF_FALSE       22

  4          18 LOAD_CONST               3 ('yes')
             21 RETURN_VALUE

  6     >>   22 LOAD_CONST               4 ('no')
             25 RETURN_VALUE
             26 LOAD_CONST               0 (None)
             29 RETURN_VALUE  

What does all this mean? Let's look at the first instruction LOAD_CONST as an example. The number in the first column (2) shows the line number in our Python source code. The second column is an index into the bytecode, telling us that the LOAD_CONST instruction appears at position zero. The third column is the instruction itself, mapped to its human-readable name. The fourth column, when present, is the argument to that instruction. The fifth column, when present, is a hint about what the argument means.

How does the Python Runtime actually work?:

With Python, it uses an interpreter rather than a compiler. An interpreter works in exactly the same way as a compiler, with one difference: instead of code generation, it loads the output in-memory and executes it directly on your system. (The exact details of how this happens can vary wildly between different languages and different interpreters.)

importlib — The implementation of import:

When reload() is executed:

Python module’s code is recompiled and the module-level code re-executed, defining a new set of objects which are bound to names in the module’s dictionary by reusing the loader which originally loaded the module. The init function of extension modules is not called a second time.

Again, please let me know if I need to edit this answer to follow etiquette.