Can packages be shared across Anaconda environments?

Question

My ~/anaconda directory is taking up too much disk space (10GB), although I have only five environments and have run conda clean. I discovered that when I try to create a new conda environment, Anaconda displays a very long list of packages to be downloaded, which seems to include a full scientific Python stack (Python interpreter, numpy, scipy, etc.). It seems that Anaconda is installing everything independently for each environment. Is this true?

The following list contains some purely speculative ways which could potentially solve the space problem:

• Can I create a "sub-environment" which "inherits" the packages of a "parent environment"?
• Can Anaconda be made to share (e.g. via symbolic links on the file system) the same packages used in different environments?
• Does the default environment have any special status in terms of package managing? I use Anaconda 2, but most of my environments use Python 3. Can I save space by switching to Anaconda 3? (This is regarding the default environment as the "parent environment" of all other environments.)
• I normally use pip to install packages, as conda install often fails. Does conda install do some smart job to reuse packages already installed somewhere else?

(Debugging information) The sizes of my four environment directories under ~/anaconda/envs are between 1.2GB and 2.6GB. Is this normal?

Answer 1

I believe the answer to your main question lies in the difference between Anaconda vs Miniconda. Anaconda includes a long list of packages that get installed automatically into each environment that you create. Miniconda creates barebone conda virtual environments (which don't contain many packages at all). Switching to Miniconda should substantially reduce the size/number of packages in your environments. Anaconda is about 2GB, while Miniconda is closer to 100MB.

Conda also uses hardlinks for packages installed vs conda install. A good description of hardlinks can be found here. They basically link dependencies across multiple environments like you've described above. Packages installed via pip are not hardlinked, so they cannot take advantage of the space savings that conda packages offer.