Code to limit GPU usage

Question

Is there a command/function/variable that can be set in CUDA code that limits the GPU usage percent? I'd like to modify an open-source project called Flam4CUDA so that that option exists. They way it is now, it uses as much of all GPUs present as possible, with the effect being that the temperatures skyrocket (obviously). In an effort to keep temps down over long periods of computing, I'd like to be able to tell the program to use, say, 50% of each GPU (or even have different percentages for different GPUs, or maybe also be able to select which GPU(s) to use). Any ideas?

If you want to see the code, it's available with "svn co https://flam4.svn.sourceforge.net/svnroot/flam4 flam4".

Answer 1

There is no easy way to do what you are asking to do. CPU usage is controlled via time-slicing of context switches, while GPUs do not have such fine-grained context switching. GPUs are cooperatively multitasked. This is why the nvidia-smi tool for workstation- and server-class boards has "exclusive" and "prohibited" modes to control the number of GPU contexts that are allowed on a given board.

Messing with the number of threads/block or blocks in a grid, as has been suggested, will break applications that are passing metadata to the kernel (not easily inferred by your software) that depends on the expected block and grid size.