How to cache in IPython Notebook?

Question

Environment:

• Python 3
• IPython 3.2

Every time I shut down a IPython notebook and re-open it, I have to re-run all the cells. But some cells involve intensive computation.

By contrast, knitr in R save the results in a cache directory by default so only new code and new settings would invoke computation.

I looked at ipycache but it seems to cache a cell instead of the notebook. Is there a counterpart of cache of knitr in IPython?

Answer 1

Unfortunately, it doesn't seem like there is something as convenient as an automatic cache. The %store magic option is close, but requires you to do the caching and reloading manually and explicitly.

In your Jupyter notebook:

a = 1
%store a

Now, let's say you close the notebook and the kernel gets restarted. You no longer have access to the local variables. However, you can reload the variables you've stored using the -r option.

%store -r a
print a # Should print 1

Answer 2

In fact the functionality you ask is already there, no need to re-implement it manually by doing your dumps .

You can use the use the %store or maybe better the %%cache magic (extension) to store the results of these intermittently cells, so they don't have to be recomputed (see https://github.com/rossant/ipycache)

It is as simple as:

%load_ext ipycache

Then, in a cell e.g.:

%%cache mycache.pkl var1 var2
var1 = 1
var2 = 2

When you execute this cell the first time, the code is executed, and the variables var1 and var2 are saved in mycache.pkl in the current directory along with the outputs. Rich display outputs are only saved if you use the development version of IPython. When you execute this cell again, the code is skipped, the variables are loaded from the file and injected into the namespace, and the outputs are restored in the notebook.

It saves all graphics, output produced, and all the variables specified automatically for you :)

Answer 3

Use the cache magic.

%cache myVar = someSlowCalculation(some, "parameters")

This will calculate someSlowCalculation(some, "parameters") once. And in subsequent calls it restores myVar from storage.

https://pypi.org/project/ipython-cache/

Under the hood it does pretty much the same as the accepted answer.