Running bash scripts in parallel

Question

I would like to run commands in parallel. So that if one fails, the whole job exists as failure. How can I do that? More specifically, I would like to run aws sync commands in parallel. I have 5 aws sync commands that run sequentially. I would like them to run in parallel so that if one fails, the whole job fails. How can I do that?

Answer 1

GNU Parallel is a really handy and powerful tool that works with anything bash

http://www.gnu.org/software/parallel/

https://www.youtube.com/watch?v=OpaiGYxkSuQ

# run lines from a file, 8 at a time
cat commands.txt | parallel --eta -j 8 "{}"