Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Do separate Anaconda environments install the same package twice, taking up twice the storage?

If I have two separate Anaconda Environments, and am installing two packages that are the same in each environment, do they install twice, and take up twice the storage?

i.e.

conda create --name myenv1

conda create --name myenv2

conda activate myenv1

conda install matplotlib

deactivate

conda activate myenv2

conda install matplotlib
like image 424
john stamos Avatar asked Aug 29 '19 20:08

john stamos


People also ask

Does conda install packages for each environment?

To automatically add default packages to each new environment that you create: Open Anaconda Prompt or terminal and run: conda config --add create_default_packages PACKAGENAME1 PACKAGENAME2. Now, you can create new environments and the default packages will be installed in all of them.

Can you have multiple conda environments?

I'd like to mention one more thing here. While you can have multiple environments that contain different versions of Python at the same time on the same computer, you can't set up 32- and 64-bit environments using the same Conda management system.

Why does Anaconda take up so much space?

One reason is that anaconda environments are completely isolated workspaces from each other with their own copy of Python. So, the more environments you have, the larger the space needed by anaconda. But the other reason is that anaconda keeps a cache of the package files, tarballs etc.


1 Answers

This was a question I had wondered about myself. No it doesn't take up twice the storage. I'm using conda version 4.7.10 in a new ubuntu 18.04 container, but you can try it with your conda version and verify the results.

Environments by default are created in the envs folder under the directory you installed anaconda in. For me that is $HOME/anaconda3. After each install you want to run du -sh $HOME/anaconda3/envs to see a summary of disk space used in human readable format.

$ du -sh $HOME/anaconda3/envs
4.0K    /root/anaconda3/envs

$ conda create --name myenv1 -y
$ conda create --name myenv2 -y
$ conda install matplotlib -n myenv1 -y

$ du -sh $HOME/anaconda3/envs
338M    /root/anaconda3/envs


$ conda install matplotlib -n myenv2 -y

$ du -sh $HOME/anaconda3/envs
357M    /root/anaconda3/envs

19M more was used, but not double.

Now the question is how do they avoid doubling the space, looking and the envs directory, I don't see any symlinks anywhere. So I looked at some files under myenv2:

$ ls -lh /root/anaconda3/envs/myenv2/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans-Bold.ttf
-rw-rw-r--. 3 root root 688K Jul  1 06:19 /root/anaconda3/envs/myenv2/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSans-Bold.ttf

The '3' after the permissions and before the file and group owner is the number of hard-links associated with a file. Normally a file has only one. Each environment must create another hard-link to the same file.

like image 153
William D. Irons Avatar answered Oct 18 '22 04:10

William D. Irons