Using multiple CUDA GPUs

Question

I'm using NVIDIA Grid K2 for CUDA programming. It has two GPUs, each with 1536 cores. Is it possible to utilize both GPUs in a single or multiple kernel calls?

Answer 1

No, it is not possible. Each GPU on a multi-GPU card like a Grid K2 is a separate CUDA device with its own memory. As such, each GPU gets its own CUDA context and must be explicitly programmed. The CUDA driver or runtime will not automagically use both devices as a single virtual device for a kernel launch.