I am trying to parallelize particle simulations with different parameters to save some time. Therefore I wanted to use GNUparallel
to run a bash script for the different parameters. The script reads a file and then performs the simulation eg :
$bash script <<< input file
However:-
$cd ~/parameter_files ls | parallel bash script <<< {}
does not work at all. I am a total newbie to Linux and GNUparallel, so hopefully someone can help.
On Linux, there are three ways to run multiple commands in a terminal: The Semicolon (;) operator. The Logical OR (||) operator. The Logical AND (&&) operator.
GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables.
xargs will run the first two commands in parallel, and then whenever one of them terminates, it will start another one, until the entire job is done. The same idea can be generalized to as many processors as you have handy. It also generalizes to other resources besides processors.
You're almost right, use double quotes around the command
ls | parallel "bash script <<< {}"
Otherwise the here string <<<
would feed the input into the parallel
command rather than each individual bash script
commands
I find this use case to be quite unusual however, since it means that basically your script is reading the filename string. If you just want to pass the files as arguments to the script, you can use
ls | parallel bash script
or if you want to pass the content of the files using standard input
ls | parallel "bash script < {}"
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With