Using a conda env created on windows for linux

Question

I am in the process of migrating a number of environments from PC to linux.

On windows I run:

C:\foo> conda env export > environment.yml

And, later, on linux, I run:

$ conda env create -f environment.yml

But I get errors like:

ResolvePackageNotFound: 
  - icc_rt=2017.0.4
  - vc=14
  - vs2015_runtime=14.0.25123
  - wincertstore=0.2
  - qt==5.9.5=vc14he4a7d60_0

This must (I assume) be a solved problem.

May someone smarter than I on this topic please let me know how they would go about this?

Thanks!

Answer 1

When exporting your environment, use the option --from-history. It will export just the libs you explicitly installed, and not the dependencies:

conda env export --from-history > environment.yml

No platform specific info will be exported. It will prevent a lot of headaches.

Usually some dependencies are platform specific. Also the default conda env export put platform specific info in the libs. This will make an environment.yml file created in Windows fail when trying to recreate it in Linux, and vice-versa.

Conda and Pip aren't really very good to perfectly recreate an environment in another machine, since they don't record all dependencies of dependencies. Usually you won't have trouble, but if you have, it will be hard to spot that a dependency of a dependency is a slightly different version.

Extra tip: always install a lib referencing its version number (e.g.: conda install pandas=1.2.1). Without the version, the command above will export the dependencies without a version, ruining the reproducibility of your environment.

But what if you created your env from a environment.yml file? Now --from-history will export the platform specific dependencies. Then grep is your friend. You will need to grep all of your import statements, see which of them are defined in your environment.yml file and just use them using the same version without platform specific info. Better to start doing it correctly using the --from-history or manually editing the file.

Extra extra tip: Python and Conda default tooling still aren't very good to create reproducible installs between platforms. Nowadays I've been successfully using Poetry for it.