What does the nVIDIA CUDA driver do exactly?

Question

What does Nvidia CUDA driver do exactly? from the perspective of using CUDA. The driver passes the kernel code, with the execution configuration (#threads, #blocks)... and what else?

I saw some post that the driver should be aware of the number of available SMs. But isn't that unnecessary ? Once the kernel is passed to GPU, the GPU scheduler just needs to spread the work to available SMs...

Answer 1

The GPU isn't a fully autonomous device, it requires a lot of help from the host driver to do even the simplest things. As I understand it the driver contains at least:

• JIT compiler/optimizer (PTX assembly code can be compiled by the driver at runtime, the driver will also recompile code to match the execution architecture of the device if required and possible)
• Device memory management
• Host memory management (DMA transfer buffers, pinned and mapped host memory, unified addressing model)
• Context and runtime support (so code/heap/stack/printf buffer memory management), dynamic symbol management, streams, etc
• Kernel "grid level" scheduler (includes managing multiple simultaneous kernels on architectures that support it)
• Compute mode management
• Display driver interop (for DirectX and OpenGL resource sharing)

That probably represents the bare minimum that is required to get some userland device code onto a GPU and running via the host side APIs.