How should conda handle header files?

Question

Trying to install a python library that depends on a C library header file. Currently, if I try:

conda install hdf5
pip install bitshuffle

Then I get a gcc error, regarding inability to locate hdf5.h

Note, conda has downloaded hdf5.h (to envs/myenv/include), and as a work-around in this case there exists an alternative channel from which conda can install bitshuffle.

Is it recommended practice to conda install gcc rather than using (or letting pip use) the system default compiler? Should conda activate prepend its include directory to an environment variable used by common compilers? Is it bad practice to combine conda and pip usage?

Answer 1

I've found the *-compiler packages (e.g., c-compiler) from Conda Forge to be relatively easy to use and they behave exactly how you are describing. That is, they automatically add the environment's include and lib folders as search locations for the compiler whenever the environment is activated. Technically, their primary use is for building Conda Forge packages, but I've had no problems using them as my compilers for local builds.

If you still wanted to use your own system-level compiler, then you could manually create your own activation scripts that mimic what the *-compiler packages do (e.g., changing CLFAGS and LDFLAGS_LD).

I would say recommended practice is to keep environments as self-contained as possible. The advantage to this is that one can create an environment definition (YAML) that deploys to multiple platforms (e.g., osx-64 and linux-64) and can build software with identical commands. Plus, if you build up code that works with the *-compilers you basically have set up an easy transition to package that code itself to a Conda package.

Finally, it should be noted that the original package in question is now on Conda Forge. The meta.yaml shows what packages are sufficient for an environment to build the package.