Jupyter autoreload fails when changing a class

Question

When using jupyter lab/notebook, most of the time, I put those 2 lines in the first cell of my notebook :

%reload_ext autoreload %autoreload 2 

Those usually allow me to modify the scripts I import and to use them without having to reimport them or to relaunch the kernel. Yesterday, I encountered an issue : I modified the script, and executing one cell on the notebook gave me an error. Restarting the kernel and redoing the imports fixed it. I investigated this issue, and tried to create a minimal example.

Let's say you have the following working directory :

+-- nb.ipynb +-- scripts |   +-- __init__.py |   +-- script1.py 

The notebook is composed of three cells :

%reload_ext autoreload %autoreload 2 

\

from scripts.script1 import Foo 

\

a = Foo(42) 

At the beginning of this experiment, script1 contains the following :

class Foo():     def __init__(self, x):         self.x = x 

Now, we execute the 3 cells of the notebook, and everything works fine. We then go to script1.py and replace its code by :

class Bar():     def __init__(self, x):         self.x = x  class Foo(Bar):     def __init__(self, x):         super().__init__(x) 

We save the file, go back to the notebook, and execute the cell containg a = Foo(42) This gives the following error :

[autoreload of script.script failed: Traceback (most recent call last):   File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/extensions/autoreload.py", line 245, in check     superreload(m, reload, self.old_objects)   File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/extensions/autoreload.py", line 384, in superreload     update_generic(old_obj, new_obj)   File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/extensions/autoreload.py", line 323, in update_generic     update(a, b)   File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/extensions/autoreload.py", line 288, in update_class     if update_generic(old_obj, new_obj): continue   File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/extensions/autoreload.py", line 323, in update_generic     update(a, b)   File "/home/user/miniconda3/lib/python3.6/site-packages/IPython/extensions/autoreload.py", line 266, in update_function     setattr(old, name, getattr(new, name)) ValueError: __init__() requires a code object with 0 free vars, not 1 ] 

Restarting the kernel or executing the import line again fixes this. Why is autoreload not working in this case?

PS : This was done in python 3.6, and my original issue with this was in python 3.7

Answer 1

1. About the problem

Why is autoreload not working in this case?

As the error log stated:

setattr(old, name, getattr(new, name)) ValueError: __init__() requires a code object with 0 free vars, not 1 

What autoreload does here:

It tries to replace the old __init__() function's (the function before code changed) code object with the the new __init__() function code object. When I check the free vars of __init__() function before and after code changed, I got the results as below:

Code to check:

Foo.__init__.__code__.co_freevars 

Results:

Before code changed: () , no free var.

After code changed: ('__class__',) , there is one free var here. That's why the error happen. It cannot replace the old object code with the new one.

2. How to solve the problem

In this case, because autoreload remain the old function object so we cannot do anything but only restart the kernel.

Hope this help.

Answer 2

The following worked for me:

from importlib import reload   reload(my_module) 

Don't know if this will work in all cases, but much easier than restarting the kernel.