Jupyter "500: Internal Server Error"; "ImportError: cannot import name ConverterMapping"

Question

I'm trying to use Jupyter notebook on a fresh install of Ubuntu 19.04. I've run through the usual installation steps, but Jupyter does not work. When I try to open a notebook, I get

500 : Internal Server Error

When I look in the terminal, I see that Jupyter has encountered the error

ImportError: cannot import name ConverterMapping

I've searched, and found questions/answers for other causes of the internal server error in Jupyter, but I cannot find a way to fix the ConverterMapping issue. I have tried uninstalling (pip uninstall ...) and reinstalling (pip install --user ...) all Jypyter-related packages, but it had no effect.

Update:

After routine system updates, the problem has returned. This indicates to me that deleting the .local directory of pip installed packages (and then reinstalling) is not necessarily a long-term fix, since it seems like automatic updates can cause the problem to re-emerge. It's also possible that some other package (installed via apt), while not explicitly relating to python, has altered the python environment in some way that breaks Jupyter notebooks.

Any ideas or suggestions would be greatly appreciated.

Answer 1

(answering my own question)

In my experience, these problems are usually caused by a compatibility issues between python packages installed via apt, and those installed via pip.

Unfortunately, I could not reconstruct the lengthy series of steps taken to install the jupyter/scipy/etc environment. Some packages must have been pulled in via apt and others via pip, leading to version conflicts.

This approach fixed the problem initially:

• Following this answer, I ran python -c "import site; print(site.USER_BASE)" to find where pip installs --user packages
• For me, this was ~/.local
• I moved ~/.local to ~/.local_disabled to eliminate all locally installed packages
• At this point, Jupyter worked again, (it must have been installed with the system python via apt).
• I've added only a minimal subset of user packages, and will keep an eye out for incompatibilities.
• I imagine avoiding pulling in Jupyter/ipython via apt might be a better way to avoid this problem in the future.

After subsequent system updates, the issue returned

This seems to confirm that the problem arises due to a mis-match between python packages installed via apt and those installed via pip. This time, to fix the issue, I:

• Uninstalled python packages that were installed via apt (but not part of the core system installed with Ubuntu). This mainly included jupyter, and many of its dependencies.
• Reinstalled Jupyter using pip install jupyter --user.
• This is working (for now).

( Please feel free to edit/comment/extend this answer. )

Answer 2

The safest solution to this is to create a virtual environment and run your jupyter from it. This clearly separates all the operations of the pip and apt. It worked for me. I use Ubuntu

To create a virtual environment, run :

1)sudo pip install --upgrade virtualenv (install virtualenv)

2)virtualenv  xyz --python=python3.7  (xyz is the name of the new virtual environment)

3) cd into the directory of xyz

4) source ./bin/activate 

5)pip install jupyter

Answer 3

I had a similar experience and found Mrule's answer. The deleting/renaming .local worked for me too but his long term solution didn't sadly. So I dug a little deep to find why the first solution worked.

Turns out which jupyter returned $HOME/.local/bin/jupyter when .local was not deleted and /usr/bin/jupyter when I did. So the problem was in the pip packages (the ones installed in .local by the pip install --user jupyter option).

I had previously did sudo pip uninstall jupyter several times before, but that doesn't remove the packages in .local (see here). You have to do pip uninstall jupyter without sudo to uninstall those in the home directory (or manually delete them). I uninstalled any other jupyter related packages in my system by finding them using pip list | grep jupyter and apt list | grep jupyter.

And then finally did a fresh install of jupyter via sudo apt install jupyter. And now it seems to be working.

Bottom line is that the packages installed via pip system wide and in the home directory (.local) and that installed via apt were conflicting somehow (I couldn't find exactly why).