Instruction Level Parallelism (ILP) and out-of-order execution on NVIDIA GPUs

Question

Do NVIDIA GPUs support out-of-order execution?

My first guess is that they don't contain such expensive hardware. However, when reading the CUDA progamming guide, the guide recommends using Instruction Level Parallelism (ILP) to improve performance.

Isn't ILP a feature that hardware supporting out-of-order execution can take advantage from? Or NVIDIA's ILP simply means compiler-level re-ordering of instructions, hence its order is still fixed at runtime. In other words, just the compiler and/or programmer has to arrange the order of instructions in such a way that ILP can be achieved at runtime through in-order executions?

Answer 1

Pipelining is a common ILP technique and is for sure implemented on NVidia's GPU. I guess you agree that pipelining doesn't rely on out-of-order execution. Besides, NVidia GPU have multiple warp schedulers from compute capability 2.0 and beyond (2 or 4). If your code has 2 (or more) consecutive and independent instructions in threads (or compiler reorders it that way somehow), you exploit this ILP from scheduler as well.

Here is a well explained question on how 2-wide warp scheduler + pipelining work together. How do nVIDIA CC 2.1 GPU warp schedulers issue 2 instructions at a time for a warp?

Also checkout Vasily Volkov's presentation on GTC 2010. He experimentally found out how ILP would improve CUDA code performance. http://www.cs.berkeley.edu/~volkov/volkov10-GTC.pdf

In terms of out-of-order execution on GPU, I don't think so. Hardware instruction reordering, speculative execution all those kind of stuff are too expensive to implement per SM, as you are aware. And thread level parallelism can fill in the gap of lacking out-of-order execution. When true dependency is encountered, some other warps can kick in and fill the pipe.