Runtime error 999 when trying to use cuda with pytorch

Question

I installed Cuda 10.1 and the latest Nvidia Driver for my Geforce 2080 ti. I try to run a basic script to test if pytorch is working and I get the following error:

RuntimeError: cuda runtime error (999) : unknown error at ..\aten\src\THC\THCGeneral.cpp:50

Below is the code im trying to run:

import torch
torch.cuda.current_device()
torch.cuda.is_available()
torch.cuda.get_device_name(0)

Answer 1

Restarting my computer fixed this for me.

But for a less invasive fix, you can also try this solution (from a tensorflow issue thread):

sudo rmmod nvidia_uvm
sudo rmmod nvidia
sudo modprobe nvidia
sudo modprobe nvidia_uvm

Answer 2

In case of PyTorch it seems to be sufficient to restart only nvidia-uvm (Unified Virtual Memory) with

sudo modprobe --remove nvidia-uvm  # same as `rmmod`
sudo modprobe nvidia-uvm

If that doesn't work, go ahead and restart the whole module with additional modprobe (--remove) nvidia as mentioned @matwilso's answer.