CUDA synchronization kernels

Question

Hi I have a doubt about programming in CUDA. I have the following code:

int main () {

    for (;;) {
        kernel_1 (x1, x2, ....);
        kernel_2 (x1, x2 ...);
        kernel_3_Reduction (x1);

    // code manipulation host_x1
    // Copy the pointer device to host
        cpy (host_x1, x1, DeviceToHost)
        cpu_code_x1_manipulation;
        kernel_ (x1, x2, ....);
    }

}

So when the copies made ​​and how do I ensure that kernel_1, kernel_2 kernel_3 and completed their tasks?

Answer 1

All operations launched on the same stream are synchronized. In the code above, all kernels will run one after another. You will have to explicitly specify streams if you need kernel_1 and kernel_2 run in parallel.