Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to run one job many times using GNU parallel?

Tags:

gnu-parallel

I can see how easy it is to run a parallel job on multiple input but is there no other way to run the same job in parallel multiple times with putting the command in a file and repeating it many times?

parallel -j+0 ::: './dosomejob.sh'

but tell parallel to run that job x amount of times in parallel using the number of available cores?

like image 373
Jon Scobie Avatar asked Dec 18 '22 22:12

Jon Scobie


2 Answers

If you have less than 10 CPU threads:

parallel -N0 ./dosomejob.sh ::: {1..10}

-N0 = insert 0 arguments.

If you want GNU Parallel to generate combinations, but you want all combinations repeated 5 times try:

parallel mycommand {1} {2} {3} {4} fixed_arg1 fixed_arg2 \
  :::: arg_file1.txt  arg_file2.txt \
  ::::+ arg_file3.txt  arg_file4.txt \
  ::: {1..5}
like image 104
Ole Tange Avatar answered Mar 24 '23 05:03

Ole Tange


Typical. I spend ages trying to work this out and when I post a question, I think I actually worked it out.

parallel -j+0 './dosomejob.sh' ::: {1..10}

Is this the correct way to do this? Is there a better parallel way?

like image 45
Jon Scobie Avatar answered Mar 24 '23 03:03

Jon Scobie