CUDA - what if I choose too many blocks?

Question

I'm still getting mad on these unknown-size matrices which may vary from 10-20.000 for each dimension.

I'm looking at the CUDA sdk and wondering: what if I choose a number of blocks too high?

Something like a grid of 9999 x 9999 blocks in the X and Y dimensions, if my hardware has SMs which can't hold all these blocks, will the kernel have problems or the performances would simply collapse?

I don't know how to dimension in blocks/threads something which may vary so much.. I'm thinking at using the MAXIMUM number of blocks my hardware supports and then making the threads inside them work across all the matrix, is this the right way?

Answer 1

The thread blocks do not have a one to one mapping with the cores. Blocks are scheduled to cores as they become available, meaning you can request as many as you want (up to a limit probably). Requesting a huge number of blocks would just slow the system down as it loads and unloads do-nothing thread blocks to the cores.

You can specify the dimensions of the grid and blocks at run time.

Edit: Here are the limits on the dimensions of the grid and the blocks, from the documentation.