Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Gnu parallel combine args from stdin and file

I am using gnu parallel command to monitor few redis endpoints and than execute all commands on new endpoints. Currently command looks like:

parallel --ungroup redis-cli -h {} monitor :::: old-server-list | cut -d  "]" -f 2 | parallel --ungroup -k ./writter.sh {}

and writter.sh

parallel redis-cli -h {} $1 ::: redis-host-1 redis-host-2 

but i am getting inconsistent time needed for commands executed on old vs new endpoints cause connections to new redis is closing and opening all the time. is it possible to pass arguments to parallel command from stdin and file at the same time? to combine each record from 'new-server-list' with same stdin input to avoid calling script?

something like:

parallel ... | cut ... | parallel redis-cli -h {<line from new-server-list>} {<stdin from pipe>}

that should be executed as:

redis-cli -h redis-host-1 stdin_output1
redis-cli -h redis-host-2 stdin_output1
redis-cli -h redis-host-1 stdin_output2
redis-cli -h redis-host-2 stdin_output2
like image 938
D. Roce Avatar asked Oct 22 '25 05:10

D. Roce


1 Answers

You could use paste to paste two columns side-by-side, one from a file and one from a command:

So, look at this:

seq 10
1
2
3
4
5
6
7
8
9
10

And a file called list.txt running the other way:

10
9
8
7
6
5
4
3
2
1

Now paste does this:

paste list.txt <(seq 10)
10  1
9   2
8   3
7   4
6   5
5   6
4   7
3   8
2   9
1   10

Which you can feed into GNU Parallel:

paste list.txt <(seq 10) | parallel -k --colsep '\t' echo {2} {1}
1 10
2 9
3 8
4 7
5 6
6 5
7 4
8 3
9 2
10 1
like image 164
Mark Setchell Avatar answered Oct 23 '25 21:10

Mark Setchell



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!