Conda-managed packages were found without entries in the package cache

Question

While trying to use conda-pack

conda pack -n myenv

the following message was displayed

Conda-managed packages were found without entries in the package cache.  This is usually due to 'conda clean -p' being unaware of symlinked or copied packages. ...

Where exactly is the package cache? Is it the packages in ...\Anaconda\Lib\site-packages ?

Does this mean packages were found in the environment I'm trying to pack (e.g. ...\Anaconda\envs\myenv\Lib\site-packages), but the packages are not in the package cache?

How should I fix this?

Answer 1

Conda Package Cache

Conda has a two-step installation process for all packages:

1. Download and unzip into package cache. By default this is Anaconda/pkgs or miniconda3/pkgs and is not associate with any particular environment. Custom locations can be specified with the pkgs_dirs configuration setting. The package cache serves as a location from which all environments can share packages in common. Sharing is done via...
2. Linking the package out to an environment location. When the pkgs_dirs and envs_dirs are on the same file system, Conda will use hardlinks to link a particular package to the environment being installed into, minimizing physical copies. When across file systems, one can additionally enable softlinks (allow_softlinks config setting), to achieve similar minimal copying, however since softlinks aren't tracked (inode count is not incremented) conda clean -p will not recognize the presence of softlinked reference and may delete packages that a softlinking environment depends on.

Hopefully, that clarifies the package cache and its purpose.

Possible Causes

Now to the problem at hand. First, it could be what is mentioned, i.e., the location of the environment is not the same as the location of the package cache, and at some point you ran conda clean -p. However, this would only happen if you aren't following default installation behavior.

The other possibility is that you have been installing things with pip. This results in packages installed into site-packages from PyPI or GitHub, but they are not Conda packages and therefore conda-pack doesn't know how to deal with it.

Suggested Workaround

In either case, I would try exporting to a YAML,

conda env export > env.yaml

and then if there is a section of pip: packages, try to replace them all with Conda versions (editing the YAML). If there isn't a pip section, then it might be the conda clean -p thing. Either way, recreate your environment using the YAML

conda env create -f env.yaml -n new_env

This should force redownloading packages missing from the cache, as well as switch to non-PyPI versions of packages, assuming you removed all the pip: packages.

I would expect conda-pack to then work on this rebuilt version of the environment.