Managing conda env in cross platform environment [duplicate]

Question

My project supposed to run on cross platform environment (Mac, Win, Linux). I've created a conda env that manage our dependencies for an easy setup. I want to ensure that everyone that want to update the enn could do that, however when I try to export the env from linux to yml file, it couldnt be install properly on Win or Mac and vise versa.

I've already tried to do the regular stuff:

1. conda env export > env.yml
conda env create --name -f env.yml

2. conda env export --no-builds > env.yml

3. https://medium.com/@Amet13/building-a-cross-platform-python-installer-using-conda-constructor-f91b70d393

4. https://tech.zegami.com/conda-constructor-tutorial-make-your-python-code-easy-to-install-cross-platform-f0c1f3096ae4

5. https://github.com/ESSS/conda-devenv/blob/master/README.rst

non of the above give me the right answer... some of the tutorials I've attached might help, but I didn't succeed to implement them correctly, and they didn't contain some important information for finishing the tutorial properly.

for instance: Regarding 3/4 - It didn't explain how to create the yml file that should construct the env.

I understood that conda supposed to work on cross platform env... It would be great if someone could help me with that.

Answer 1

Conda Envs are Not Inherently Cross-Platform

Sorry, but what you're asking for is simply not a thing. Conda can serialize an environment's package information to a YAML (great for reproducibility), but it can't guarantee that it will be cross-platform. In fact, many packages, especially ones with non-Python code, require different underlying build tools as dependencies, so what you're asking for will never be satisfied.

Explicit Specs Only

The closest you can get these days is to limit your environment.yaml to only include explicit specs that have gone into creating your environment by using the --from-history flag. This feature requires Conda v4.7.12 or later.

conda env export --from-history > environment.yaml

This will generate a YAML that only includes the packages that have been explicitly requested in the history of the env, e.g., if your history goes...

conda create -n foo python=3.7 numpy
conda install -n foo pandas scikit-learn

Then the result of conda env export -n foo --from-history will be something like

name: foo
channels:
  - defaults
dependencies:
  - python=3.7
  - numpy
  - pandas
  - scikit-learn
prefix: /your/conda/dir/envs/foo

This way, you can leave out all the other dependencies that may turn out to be platform-specific.

---

I'm Still Seeing a Ton of Packages?!

I've noticed that if one ever uses the --update-deps flag in an env, it adds every dependency to being an explicit spec. This is rather unfortunate. If this is the case, I'd suggest recreating the env using your legitimate specs and avoid that flag in the future. Searching through your command history might be useful in compiling that legitimate spec list.