How to remove cuInit failed: unknown error in CUDA (PyCuda)

Question

I have CUDA installed then I tried the following:

In [1]: import pycuda.driver as cuda

In [2]: cuda.init()
---------------------------------------------------------------------------
Error                                     Traceback (most recent call last)
<ipython-input-2-2845c9c0ab3c> in <module>()
----> 1 cuda.init()

Error: cuInit failed: unknown error

How can resolve the error?

I already have this installed:

$ which nvidia-modprobe
/usr/bin/nvidia-modprobe

My other info:

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2015 NVIDIA Corporation
Built on Tue_Aug_11_14:27:32_CDT_2015
Cuda compilation tools, release 7.5, V7.5.17

And this:

$ uname -a
Linux foobar1 4.10.0-28-generic #32~16.04.2-Ubuntu SMP Thu Jul 20 10:19:48 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

And this is the information for the nvidia-smi

$ nvidia-smi
Mon Nov 19 15:56:43 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.48                 Driver Version: 410.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro P400         Off  | 00000000:AF:00.0 Off |                  N/A |
| 34%   45C    P0    N/A /  N/A |      0MiB /  1999MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Answer 1

cuInit failed: unknown error is often the result of nvidia-uvm kernel module not being loaded. I've been periodically running into this issue on Ubuntu.

sudo nvidia-modprobe -u should fix the problem. That is, until you reboot. Then you will need to do it again.

Another workaround is to run your failing application as root once. AFAICT it in this case CUDA runtime will attempt to load the missing module (and will most likely succeed at that, because we're running as root).