How to clean iPython environment so I can start over with Jupyter and Python 3.x?

Question

Over the past couple years I've pip installed lots of things without knowing what I was doing. All of a sudden, I am getting this error when I run a snippet in a Python 2.7 Jupyter Notebook:

ImportError: No module named matplotlib

When I run this in Terminal: which -a python python 3... I get:

Sams-MacBook-Pro-2:~ sambrand$ which -a python python3
//anaconda/bin/python
/usr/local/bin/python
/usr/bin/python
/usr/local/bin/python

I think this is showing that I have Python in a few places, which might explain why my standard libraries (eg. matplotlib) aren't being found in the Python environment that I want to use.

My ideal result:

Ultimately, I just want to clean up my environment and start from scratch so that I can use Keras, Tensorflow, and all Anaconda packages with Python 3 as my default when I open a Jupyter Notebook.

Additionally, I'd like any subsequent pip install from the terminal to update what I'm using in Jupyter.

From what I gather, this kind of bewilderment about Python environment is a fairly common issue, so I hope this post proves helpful.

Answer 1

Having a lot of python versions on the same machine is quite a common situation. To clean up your python you need to do the following steps:

0. Keep in mind that as for OSX 10.15, "native" python version in still 2.7. It lives in /usr/bin/. (Actual path is /System/Library/Frameworks/Python.framework/Versions/2.7/bin/ but it's the same thing.) If you want to roll back to the clean system python environment, you need to delete everything else.

1. First of all you need to find all other pythons. On my machine it was like this:

   $ which -a python python3
   /Users/yura/anaconda3/bin/python # <- Anaconda
   /opt/local/bin/python            # <- ports
   /opt/local/bin/python            # <- ports
   /opt/local/bin/python            # <- ports
   /opt/local/bin/python            # <- ports
   /usr/bin/python                  # <- "native"
   ~/anaconda3/bin/python3          # <- Anaconda
   /Library/Frameworks/Python.framework/Versions/3.5/bin/python3 # <- python3 for OSX
   /usr/local/bin/python3           # <- python3 for OSX```
   
   

As you may see, I had installed more stuff from mac ports, I had python3 for Mac downloaded and installed by hands, and also I had badly installed Anaconda (it is not seen from here, but that installation had wrong access rights). Also, you may have something from homebrew, which I don't use. It will also appear in /usr/local/bin. Well, let's get started!

3. remove Anaconda: $ rm -rf ~/anaconda3

4. remove everything from /Library/Frameworks/Python.Framework/:

   $ sudo rm -rf /Library/Frameworks/Python.Framework/

5. remove everything from /Applications/Python*/, which you may installed manually:

   $ sudo rm -rf /Applications/Python*

6. remove all the symlinks from /usr/local/bin:

   $ sudo rm /usr/local/bin/python*

7. remove all packages installed by pip in ~/Library/Python/: '

   $ rm -rf ~/Library/Python/

8. And finally, you may also remove all port-related files, they are in /opt/local/bin/python*. WARNING: it may break some other port packages! So the most accurate way to do it is to use port itself (but you may skip this step in order to leave untouched the other software from ports):

   $ sudo port uninstall python*

9. That's it! Now you have only a system python2.7. You can download Anaconda and install it:

   $ sh Anaconda3-*-MacOSX-x86_64.sh

10. Now you have a new python3. To check this open a new terminal and try:

    $ python --version
    Python 3.6.5 :: Anaconda, Inc.
    

11. And matplotlib and all the other scientific stuff like pandas etc is already there:

    $ python -c "import matplotlib as mpl; print(mpl.__version__)"
    2.2.2
    

Answer 2

The best practice to keep it 'clean' is to use virtual environments. You can use conda to do it.

From terminal run:

conda create -n envs_name python=3.6

for example.

After that you need to activate it, this is like saying "do the following only in my virtual environment and not on the global one":

source activate envs_name
pip install keras
pip install tensorflow
pip install ipykernel

The ipykernel let's you manage your environments inside Jupyter.

It's really easy and convenient.

Answer 3

It took me hours, and I haven't "cleaned" my environment, but I was able to get Python 3, Keras, Tensorflow, and Anaconda running in a Jupyter notebook by following these steps:

1. Delete Anaconda from computer
2. Install Anaconda from the web
3. Install Python 3 from the web
4. Pip install Keras from Terminal (for some reason Keras wasn't shown in Anaconda Navigator, as recommended in this post)
5. Pip install Tensorflow from Terminal
6. Open Jupyter Notebook from Terminal