Cannot install netCDF4 by pip on windows

Question

I just installed 'vanilla' python on windows 10 and tried to install netCDF4, only to find it gives an error saying that: "ValueError: did not find HDF5 headers"

This is Python 3.7 and versions are as follows:

py -m pip list
Package            Version
------------------ -------
attrs              19.1.0
backcall           0.1.0
bleach             3.1.0
bokeh              1.3.4
colorama           0.4.1
cycler             0.10.0
Cython             0.29.13
decorator          4.4.0
defusedxml         0.6.0
entrypoints        0.3
h5py               2.9.0
ipykernel          5.1.2
ipython            7.8.0
ipython-genutils   0.2.0
ipywidgets         7.5.1
jedi               0.15.1
Jinja2             2.10.1
jsonschema         3.0.2
jupyter            1.0.0
jupyter-client     5.3.1
jupyter-console    6.0.0
jupyter-core       4.5.0
kiwisolver         1.1.0
MarkupSafe         1.1.1
matplotlib         3.1.1
mistune            0.8.4
nbconvert          5.6.0
nbformat           4.4.0
notebook           6.0.1
numpy              1.17.1
packaging          19.1
pandas             0.25.1
pandocfilters      1.4.2
parso              0.5.1
pickleshare        0.7.5
Pillow             6.1.0
pip                19.2.3
pip-review         1.0
pivottablejs       0.9.0
prometheus-client  0.7.1
prompt-toolkit     2.0.9
Pygments           2.4.2
pyparsing          2.4.2
pypl               40.10
pyrsistent         0.15.4
python-dateutil    2.8.0
pytz               2019.2
pywinpty           0.5.5
PyYAML             5.1.2
pyzmq              18.1.0
qtconsole          4.5.5
scipy              1.3.1
seaborn            0.9.0
Send2Trash         1.5.0
setuptools         41.2.0
six                1.12.0
svgwrite           1.3.1
terminado          0.8.2
testpath           0.4.2
tornado            6.0.3
traitlets          4.3.2
virtualenv         16.7.5
wcwidth            0.1.7
webencodings       0.5.1
widgetsnbextension 3.5.1
xarray             0.12.3

When I try to install netCDF with pip, it gives an error. The whole error message is:

>py -m pip install netCDF4
Collecting netCDF4
  Using cached 

 https://files.pythonhosted.org/packages/64/2e/abc0bce095ab5a3b8374f052ace2509a031fd7633b23917e557487067225/netCDF4-1.5.2.tar.gz
    ERROR: Command errored out with exit status 1:
     command: 'c:\python37-32\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\***\\AppData\\Local\\Temp\\pip-install-374v_ecc\\netCDF4\\setup.py'"'"'; __file__='"'"'C:\\Users\\***\\AppData\\Local\\Temp\\pip-install-374v_ecc\\netCDF4\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\
    Complete output (16 lines):
    reading from setup.cfg...

        HDF5_DIR environment variable not set, checking some standard locations ..
    checking C:\Users\***...
    checking /usr/local ...
    checking /sw ...
    checking /opt ...
    checking /opt/local ...
    checking /usr ...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\setup.py", line 397, in <module>
        _populate_hdf5_info(dirstosearch, inc_dirs, libs, lib_dirs)
      File "C:\Users\***\AppData\Local\Temp\pip-install-374v_ecc\netCDF4\setup.py", line 345, in _populate_hdf5_info
        raise ValueError('did not find HDF5 headers')
    ValueError: did not find HDF5 headers
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Please tell me what to do or what information you need. Thank you.

Answer 1

TL;DR: Check if there's an issue with the present h5py wheels if you run into this problem

I've just run into almost the exact same problem as the OP this afternoon and spent the last hour banging my head against a wall.

My story:

1. I am using Python 3.11.1
2. Found this thread. I had seemingly the same issue as the OP
3. I considered installing the build tools for visual studio but noticed it was going to be ~5 GB...
4. Decided to investigate further
5. Found this GitHub issue demonstrating it's an issue with the h5py wheel for 3.11.1
6. Installed netCDF4 on a different computer with Python 3.8.3 without any issues
7. I will be using Python<=3.10 until h5py wheel is sorted

So was the C++ compiler the real solution to the OP's issue? Well I checked and it turns out, there was another issue open for wheels not being prepared correctly for h5py back in September 2019 which was closed around the time of the OP's first post. Though, there seems to be a ~23 hour gap in the timeline (I think I've accounted for timezones correctly). Perhaps the C++ compiler was able to resolve the issue introduced by the faulty wheels?

Answer 2

I resolved the problem by myself. I needed to install C++ compiler manually since it is not included in vanilla python package (of course).

I downloaded and installed 'build tools for visual studio' from the site below: https://visualstudio.microsoft.com/downloads/ and this solved the problem.