Jupyter Notebook: (OperationalError('disk I/O error',))

Question

I'm getting the following error on running an existing Jupyter Notebook file

The history saving thread hit an unexpected error (OperationalError('disk I/O error',)).History will not be written to the database.

Never the less, the remaining code in the notebook executes properly after that. However, when I'm trying to create a new notebook file this error is not letting me create it and I get the following error:

Error while saving file: python_notebook_ollie/Untitled2.ipynb disk I/O error
Traceback (most recent call last):
  File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/filemanager.py", line 421, in save
    self.check_and_sign(nb, path)
  File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/notebook/services/contents/manager.py", line 440, in check_and_sign
    self.notary.sign(nb)
  File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 449, in sign
    self.store.store_signature(signature, self.algorithm)
  File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 207, in store_signature
    if not self.check_signature(digest, algorithm):
  File "/work/ollie/muali/miniconda3/lib/python3.6/site-packages/nbformat/sign.py", line 241, in check_signature
    self.db.commit()
sqlite3.OperationalError: disk I/O error
[W 15:08:48.093 NotebookApp] Unexpected error while saving file: python_notebook_ollie/Untitled2.ipynb disk I/O error
[E 15:08:48.094 NotebookApp] {
  "Host": "localhost:15695",
  "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0",
  "Accept": "application/json, text/javascript, */*; q=0.01",
  "Accept-Language": "en-US,en;q=0.5",
  "Accept-Encoding": "gzip, deflate",
  "Referer": "http://localhost:15695/notebooks/python_notebook_ollie/Traj_case_study.ipynb",
  "Content-Type": "application/json",
  "X-Xsrftoken": "2|96ae1fed|bde68f8333dd4b46b900b02246747fa4|1520496687",
  "X-Requested-With": "XMLHttpRequest",
  "Content-Length": "19",
  "Cookie": "_xsrf=2|96ae1fed|bde68f8333dd4b46b900b02246747fa4|1520496687; username-localhost-15695=\"2|1:0|10:1520518043|24:username-localhost-15695|44:NjE5MWZlMWIzMjdmNGE2N2FlZmQ3NmE3NzRlNmNiZmQ=|8062f4c541e5dcef0d8b4a2d7e75cc59f1d27197ac4633b9da64b7bb94aae7a4\"",
  "Connection": "keep-alive"
}  

I tried to follow this answer and deleted the history.sqlite file but, still I'm getting the disk I/O error.

Some threads on github say NFS based mount can create some problems but in the past it was working fine until I upgraded my matplotlib package. I've now downgraded that package but I still can't get rid of the error.

Edit: The first error displayed on the terminal while opening an .ipynb file is

The signatures database cannot be opened; maybe it is corrupted or  

encrypted. You may need to rerun your notebooks to ensure that they are trusted to run Javascript. The old signatures database has been renamed to ~/.local/share/jupyter/nbsignatures.db.bak and a new one has been created.

Answer 1

I was able to make it work so I'll post what I did. I think the cause of the problem is that SQLite locking doesn't work reliably on NFS filesystems as described in this link.

The answer is also given there and I'll write the steps in more detail here:

Basically, one has to create a new configuration for ipython or modify an existing one. The config file named as ipython_config.py can be found in ~/.ipython/profile_default

If the config file doesn't exist by default then one can create it by typing the following in your home directory

ipython profile create 

This will create the configuration files under ~/.ipython/profile_default. Open ipython_config.py and add the following lines:

c = get_config() #gets the configuration
c.HistoryManager.hist_file='/tmp/ipython_hist.sqlite' #changes history file writing to tmp folder

Save it and re-open the kernel and it should work now. More details on modifying configurations can be found here

If the configuration file already exist then just type in the terminal

ipython --HistoryManager.hist_file='/tmp/ipython_hist.sqlite'

and that should also work.

Answer 2

I followed the steps above and I'm not sure what went wrong, but it completely crippled Jupyter lab (print("hello") was taking ~30 seconds to render).

In my case, I believe the original problem was caused because the history was corrupted (~/.ipython/profile_default/ had a file called history-corrupt-* with today's timestamp). I cleared the history by deleting ~/.ipython/profile_default/history.sqlite per Ankit's answer here and that solved the problem.

N.B: I also deleted the config files created during the procedure above.