TensorFlow on Windows: "Couldn't open CUDA library cudnn64_5.dll"

Question

Tensorflow just released windows support. I installed the gpu version and CUDA 8.0 and python 3.5. However, after I import the tensorflow I got the following error:

>>> import tensorflow
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cublas64_80.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:119] Couldn't open CUDA library cudnn64_5.dll
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\cuda\cuda_dnn.cc:3459] Unable to load cuDNN DSO
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library cufft64_80.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library nvcuda.dll locally
I c:\tf_jenkins\home\workspace\release-win\device\gpu\os\windows\tensorflow\stream_executor\dso_loader.cc:128] successfully opened CUDA library curand64_80.dll locally

Can someone help? Thanks!

Answer 1

TL;DR: To use cuDNN with TensorFlow, the file cudnn64_5.dll must be in a directory that is in your %PATH% environment variable. Note that cuDNN is a separate download from CUDA, and you must download version 5.1 of cuDNN.

First of all, note that cuDNN is not distributed with the rest of the CUDA toolkit, so you will need to download it separately from the NVIDIA website. On Windows, it is distributed as a ZIP archive, so you must extract it and find the directory containing cudnn64_5.dll. For example, if you extract it to C:\tools\cuda, the DLL will be in C:\tools\cuda\bin\cudnn64_5.dll. Finally, you can add it to your path by typing the following at the command prompt:

C:\> set PATH=%PATH%;C:\tools\cuda\bin
C:\> python
...
>>> import tensorflow as tf

Answer 2

Tried pip3 install --upgrade tensorflow after tensorflow-gpu and it worked fine.

I think it's an issue only when trying pip3 install --upgrade tensorflow-gpu directly.

Answer 3

In addition to the answers above make sure that you've downloaded the supported version of cuDNN. Currently TensorFlow supports the older cuDNN v.5.1 while there is a newer cuDNN 6.0 available on Nvidia site. I had such errors with 6.0. When I rolled back to 5.1 everything worked.

• Check TensorFlow requirments here: https://www.tensorflow.org/install/install_windows
• Download the supported version of cuDNN from here: https://developer.nvidia.com/rdp/cudnn-download

Answer 4

I had this problem and it took me several attempts to resolve it. This answer applies to Python 64 on Windows 64 I also have VS2017 installed with Python 3.6

From a clean Windows 64 machine Install Visual Studio 2015 (note: NOT vs2017 -- at least not yet). The community edition is free. Make sure you install the C++ compiler. You will need this to compile future python libs.

This will also clean up any problems with msvcp140.dll or msvcrt*.dlls. Alternatively you can install the VC Redistributable (but i recommend installing VS2015 instead as this will allow you to compile and install future python libraries).

Next, install VS2017 and this time also select Python and Data Learning (the scikit). This will default to installing Anaconda with Python 3.6 (built with VS2015). There are also some useful features

*Also make sure you have a compatible Nvidia card (see previous answers)

Then make sure you have the latest Nvidia drivers installed on your computer.

Then make sure you have installed the Nvidia libraries mentioned by Google and others including the Cuda bins.

*As of tensorflow 1.2, Python 3.6 is supported so the notes on creating a Python 3.5 environment are no longer necessary

There are a few problems i ran into with tensorflow 1.2. I also tried tensorflow 1.31rc2 in my environment

Problem #1 -- Firewalls (for those behind a firewall)

This will prevent installations via "pip install" To fix this, add --trusted-host pypi.python.org

For example

pip install tensorflow-gpu --trusted-host pypi.python.org

Problem #2 -- Upgrading to numpy 1.13.1

Installing tensorflow will upgrade to an incompatible version of numpy 1.13.1 (at least on my windows machine). To fix this, download the wheel at http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy look for numpy-1.13.1+mkl-cp36-cp36m-win_amd64.whl

Install this wheel before installing tensorflow. This will stop tensorflow from installing an incompatible version of the numpy 1.13.1 package

NOTE: see how to install wheels in other posts (pip install fullpath_of_wheel)

NOTE: if you have already installed tensorflow, make sure you force an upgrade using the --upgrade option with pip install)

Problem #3 -- Nvidia cuda packages in different locations

To test if you have the proper CU*.DLL packages use the "where" command (from a C:\ command prompt)

where cublas64_80.dll

where cudnn64_5.dll

where cufft64_80.dll

where nvcuda.dll

where curand64_80.dll

where cusolver64_80.dll

If your machine is like mine, you will find those dlls in multiple locations and sometimes they do not even exist. For instance, cublas64_80.dll was found in my c:\program files\Anaconda3 directory nvcuda.dll in my c:\system32\windows and so on. If you have matlab installed, it will have its own version. CNTK has its own versions also. Nvidia will put them in another directory. This is yet another problem. As mentioned by others, some of the DLLs you need are provided by Nvidia in a zip file.

Instead of trying to fix up your paths, I recommend trying this first instead

Collect the 6 dlls mentioned above and place them in ONE directory such as c:\tfexperiment

Then cd into c:\tfexperiment

run python.exe from this location. Windows will now look for the dlls in the current path first

now once python loads type in

import tensorflow as tf

it should work for you (hopefully). This was the only way I was able to get it to work on my machine. If you get this far, you can simply add c:\tfexperiment as your first path in the path environment variable. Or you can figure out the correct path order.

If it STILL does not work, you can take it one step further by downloading procmon.exe from Microsoft. Run procmon.exe. Filter on the executable python.exe (sorry i dont have time to explain how to use procmon). Now go back to your python prompt and type "import tensorflow as tf" again. procmon should have many lines of information. You may want to filter on loadimage. This will tell you what dlls it is loading. Note that .pyd extensions are also DLLs. The last .dll that loaded (or failed to load) is probably the one that caused problems.