Strict load balancing of multiple .NET processes

Question

I have a multi-process .NET (F#) scientific simulation running on Windows Server 2008 SE and 64 processors. Each time step of the simulation oscillates from 1.5 sec to 2 sec. As each process must wait for other processes, the overall speed is the speed of the slowest process (2 sec * number of iterations). Therefore, I need to reduce the oscillation of the processes as much as possible.

Is there any way how to force a set of processes to have the exactly same "computational time" available for their computations?

Answer 1

Is it possible for you to paralelize the 2 second series, so that you have multiple "branches" of the simulation occuring in parallel?

Example: Suppose that this is 1 simulation with 4 processes. Process 1 takes 2 seconds, so you cannot finish until process 1 completes.

process1---------------------------------------------- (2 sec)
process2-------- (0.5 sec)
process3---- (0.25 sec)
process4---------------------------- (1 sec)

You have a lot of idle time in there where most of your processes are waiting on process 1.
For the work you are trying to do, is it feasible to have more than 1 of these sets running at the same time? If so, then you could utilize your idle cores by working on other simulations while they are waiting for your longer running process to finish.