Run jobs in parallel

Question

I would like to run a large number of intensive processes in parallel, where I loop over different parameters with for loops. A number of answers to similar questions mention that running processes in parallel can be done with xargs, but none of them seem to mention if or how this can be done if the parameters change for each command.

As an example (pseudo code):

for paramA in 1 2 3
  for paramB in 1 2 3
    ./intensiveCommand $paramA $paramB
  end
end

I would like to parallellize intensiveCommand

Or is there an easier way then using xargs?

Answer 1

You can use GNU parallel. It has a --load option avoid overloading the computer.

parallel --load 100% ./intensiveCommand ::: 1 2 3 ::: 1 2 3