How to check if GPU memory is available using PyOpenCL

Question

I would like to know if there is a way to check how much GPU memory is available before a function uses it. I have code that often uses 1.5 GB of GPU memory or more, and if something else is using the GPU when my program wants to use it, I get a MemoryError exception or something similar.

I would like to implement some sort of code so that I can check to see if the GPU has enough memory available, and if it does, go ahead and run, but if not, wait until it IS available.

(Preferably, I would like to check before trying to use the GPU rather than using a try-except loop and just retrying if it fails)

I checked the PyOpenCL documentation to see if there was something relevant under device_info, but I couldn't find any actual descriptions.

Answer 1

This is not possible, and is actually a limitation of OpenCL, not just PyOpenCL. See here.

On NVIDIA devices, you can use nvidia-ml-py. Then you can do something like this:

from pynvml import *
nvmlInit()
for i in range(nvmlDeviceGetCount()):
    handle = nvmlDeviceGetHandleByIndex(i)
    meminfo = nvmlDeviceGetMemoryInfo(handle)
    print("%s: %0.1f MB free, %0.1f MB used, %0.1f MB total" % (
        nvmlDeviceGetName(handle),
        meminfo.free/1024.**2, meminfo.used/1024.**2, meminfo.total/1024.**2))
nvmlShutdown()