Reloading a Python extension module from IPython

Question

Using Cython, I am developing an extension module which gets build as an .so file. I then test it using IPython. During development, I frequently need to make changes and rebuild. I also need to exit the IPython shell and reenter all commands. Reimporting the module with

import imp
imp.reload(Extension)

does not work, the code is not updated. Is there a way for me to avoid restarting the IPython shell after I rebuild the module?

Answer 1

C extensions cannot be reloaded without restarting the process (see this official Python bug for more info).

Since you are already using IPython, I might recommend using one of the two-process interfaces such as the Notebook or QtConsole, if it's acceptable to you. These allow you to easily restart the kernel process, which allows to you load the module anew. Obviously, this isn't as convenient as reload for a Python module because you have to re-execute to get back to the same state. But that is not avoidable, so the point is to mitigate the inconvenience.

I find the notebook interface the most convenient for developing extensions, because it provides the easiest way to get back to the same state:

1. rebuild the extension
2. restart kernel
3. Run All to re-execute the notebook

and you are back to the same state with the new version of the extension. Mileage may vary, depending on how costly your interactive work is to re-run, but it has served me well.