How to force openMP to run iterations in specific order

Question

I am using openMP to run instances of my simulation in parallel.

#pragma omp parallel for private(part) shared(P,lfcc,temp)
for (part = 0; part < P->Parts; part++)

Part of the loop checks if the output file with index number "part" already exist (so i can run the simulation for a certain value of "Parts", and later increase it without overwriting the existing results). However, this requires the iteration to run in order. That is, for n threads, it should first run simultaneously parts (1)-(n), followed by parts (n+1)-(2n) and so on. right now (with 3 threads running in parallel, and "parts" set to N), the behaviour is different, running first parts (0),(N/3),(2N/3) followed by (1),(N/3+1),(2N/3+1) and so on.

Suppose now, i initially wanted 30 parts which were all completed. Then i decide i need more parts and change "Parts" to 45. Then the first threads gets parts (1)-(15), the second (16)-(30) and the third (31-45). The first two threads quickly find out that all their assigned parts had already been completed and will leave the last thread to work alone (if i put a barrier clause before the program terminates).

One simple solution is to let the "part" variable start not with 0 but with m+1, where m is the number of previously completed parts. But i was wondering if it was possible to force openMP to run the iteration in the order showing in bold above.

Answer 1

You can change the size of the iteration blocks each thread gets to 1 within the schedule clause, e.g. schedule(static,1). With 3 threads the first one would process iterations 0, 3, 6, 9 and so on, the second thread would process iterations 1, 4, 7, 10 and so on, and the third one would process iterations 2, 5, 8, 11 and so on. You still need to synchronise somewhere in the loop since there is no guarantee that threads would execute all steps at the same time and at the same speed (you can put a barrier at the end of each iteration to synchronise before the next block of iterations starts).

Another solution is to use the OpenMP tasking construct. With it you can run a big loop in one thread, generating computational tasks. You can put checks for the existence of the output file inside this loop and create new tasks only if needed (e.g. the output file does not exist):

#pragma omp parallel
{
    ...
    #pragma omp single
    for (part = 0; part < P->Parts; part++)
    {
        if (!output_file_exists(part))
           #pragma omp task
           {
              ... computation for that part ...
           }
    }
    #pragma omp taskwait
    ...
}

Hope I've understood your problem correctly.