Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GNU parallel with variable sequence?

I want to run a program prog in parallel using GNU's parallel, with an argument that takes a value in a sequence. For example:

parallel prog ::: {1..100}

However, I don't know the upper bound of the sequence in advance, so I would like to be able to do something like:

parallel prog ::: {1..$x}

where $x is a number that I'll compute somewhere. How can I achieve this?

like image 584
becko Avatar asked Jun 03 '15 20:06

becko


1 Answers

Assuming the seq program (or something like it) is available,

parallel prog ::: $(seq 1 $x)

If not, you can fake it:

parallel prog ::: $(for ((i=1; i < x; i++)) do; echo $i; done)

As Ole points out, if $x is large, then the resulting sequence of numbers may be too large to fit on the command line. In that case, use either of the two methods above to feed the arguments to parallel via standard input:

seq 1 "$x" | parallel prog
for ((i=1; i<x; i++)) do; echo "$i"; done | parallel prog
like image 113
chepner Avatar answered Oct 21 '22 17:10

chepner