conda-forge: Why does Conda inconsistently want to downgrade NumPy?

Question

I'm trying to prefer packages from CondaForge (for availability and compatibility). However, Conda seems to want to prefer versions of core libraries (e.g. NumPy) from other channels.

For example, when I try to install a new library, Conda offers to downgrade NumPy, but if I ask for the same library and NumPy, Conda no longer advises downgrade. Why?

$ conda install -c conda-forge beautifulsoup4
The following NEW packages will be INSTALLED:
    beautifulsoup4: 4.6.3-py36_0 conda-forge
The following packages will be UPDATED:
    numpy-base: 1.14.3-py36h0ea5e3f_1 --> 1.15.0-py36h3dfced4_0
The following packages will be DOWNGRADED:
    blas: 1.1-openblas conda-forge --> 1.0-mkl
    numpy: 1.15.1-py36_blas_openblashd3ea46f_1 conda-forge [blas_openblas] --> 1.15.0-py36h1b885b7_0
    scikit-learn: 0.19.2-py36_blas_openblasha84fab4_201 conda-forge [blas_openblas] --> 0.19.1-py36hedc7406_0
    scipy: 1.1.0-py36_blas_openblash7943236_201 conda-forge [blas_openblas] --> 1.1.0-py36hc49cb51_0
Proceed ([y]/n)? n

$ conda install -c conda-forge beautifulsoup4 numpy
The following NEW packages will be INSTALLED:
    beautifulsoup4: 4.6.3-py36_0 conda-forge
Proceed ([y]/n)? y

Is there a preference that will give better behaviour?

Answer 1

This is happening because you have two channels (conda-forge and defaults), both of which contain NumPy (and its dependencies), but possibly at different version / build numbers.

For example, say you want to install SciPy (which depends on NumPy) and the state of the world is:

• conda-forge: NumPy v1.14 and SciPy v1.0
• defaults: NumPy v1.15 and SciPy v1.0

And you have conda-forge above defaults in your channel order. If you say conda install scipy, then Conda will grab SciPy from conda-forge (because it is the highest version number). While scanning SciPy's dependencies it will notice that there is a newer version of NumPy available on defaults. Thinking it is being helpful, Conda will install the newer version of NumPy from defaults even though you already had it installed from conda-forge. If there are packages that NumPy depends on that then have to be downgraded to make this work, so be it.

By saying instead, conda install scipy numpy or conda config --add pinned_packages conda-forge::numpy you are skipping the dependency lookup for that part of the graph, that then causes the Conda solver to hop to a different channel.

This is a relatively simple example and definitely doesn't cover all of the weird edge cases that come up on a daily basis.

That said, Conda v4.6 (which isn't out yet) will add a notion of "strict channel priority." This will ensure that the solver looks for packages in the channel order given first and only hops to a different channel if a dependency cannot be found.

This will solve a lot of these rampant upgrade/downgrade issues we have all been living with.

Answer 2

This doesn't really explain why it was happening, but a fix is:

$ conda config --add pinned_packages conda-forge::numpy

Answer 3

You can read the conda-forge documentation http://conda-forge.org/docs/user/tipsandtricks.html#multiple-channels

To see your channels and priorities $conda config --describe channel_priority

The solution is to add the conda-forge channel on top of defaults in your .condarc file when using conda-forge packages and activate the strict channel priority with:

$ conda config --set channel_priority strict

Here is how a .condarc file would look like:

$ cat .condarc channel_priority: strict channels: - conda-forge - defaults

In addition to the channel priority we recommend to always install your packages inside a new environment instead the root environment from anaconda/miniconda. Using envs make it easier to debug problems with packages and ensure the stability of your root env.