Difference between conda and pip installs within a conda environment

Question

I seem to be asking myself this question a lot, having recently switched to using conda environments (Anaconda), but I end up Googling and not getting too far.

I now run all my projects within their own conda environments, as I like to keep everything as separate and with as little dependencies on other programs as possible. For example, a recent environment:

conda create -n RL numpy tensorflow-gpu

Then I activate the environment, and realise "Oh - I forgot to install gym". In this case, this is only available in the PIP package manager, and so I simply type pip install gym. But in other cases, where the package exists within conda and pip, what is the best way to install it?

conda install package pip install package

Or in other words - what is the difference?

To provide the full picture, I'm running everything in Ubuntu 16.04, and switch between python 2 and 3 depending on the project. So some of my conda environments are in python 2, some are python 3. I've found that sometimes a pip3 install is required for python 3, but not always - why is this?

Secondly, my path links to the python setup in my Anaconda3 directory.

My current idea is that if I install via conda, it installs directly to my environment, but via pip it installs to my anaconda3 site-packages, making it available to all conda environments under my Anaconda3 directory. If this is the case, this means that if I pip install gym in one conda environment, it should also be available in all others - but this isn't the expected behaviour of environments as far as I am aware.

Please feel free to correct my assumptions and knock some sense into me!

Answer 1

For my understanding of Conda that, it manages for you all the dependencies. For example if you have a package (like pandas) that requires another package (like numpy), it will download both (after warning you).

Where conda is becoming handy is that sometimes a specific package requires a specific version of another one (4.3 or later for example) and they can be conflicts between the packages. The requirements and conflicts define a mathematical problem that can be solved thanks to a SAT solver.

You can find informations and link about that here: https://www.continuum.io/blog/developer/new-advances-conda-0

So each time you are installing a new package, it will upgrade (or sometimes downgrade if conflicts) other packages to ensure the functionning of each package. Personnaly, I go with conda and use pip only when the package is not managed by conda

Another link if you are interested by conda: https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/

About pip3, it is the naming used when you have both Python 2 and Python 3 installed, to avoid conflicts in the command. In a python 3 environment, the command pip will be equivalent to pip3.

For the behavior of pip, I can confirm that the installation is done only in the active environment and is not available to the other ones