Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to pass quoted args to GNU Parallel

I need to pass some text that includes whitespace and other characters to a script that's being run by GNU Parallel.

Here is a very simple example:

$ seq 1 3 | parallel echo "Quoted ' (text)"

The above example will output this:

sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file    

However, if I do this everything works:

seq 1 3 | parallel echo "\"Quoted ' (text)\""

I happen to be running this from a python script, so before passing the arguments I'm double quoting them in the script like this:

args = ["Some arg", "Another arg", "etc."]
args = ' '.join(pipes.quote(pipes.quote(arg)) for arg in args)

But it doesn't seem like a clean solution.

Does anybody know of a better way to pass arguments to GNU Parallel?

Thanks!

like image 273
chaimp Avatar asked Nov 22 '11 16:11

chaimp


2 Answers

zsh-4.3.12[sysadmin]% print -l {1..3} | 
  parallel -q echo "Quoted ' (text)"
Quoted ' (text) 1
Quoted ' (text) 2
Quoted ' (text) 3

As described by @mortehu:

Arguments passed to commands through parallel are expanded by the shell twice: once in the invocation of parallel, and once when parallel runs your command. -q prevents the second shell expansion.

like image 163
Dimitre Radoulov Avatar answered Nov 14 '22 23:11

Dimitre Radoulov


There is a whole section in the man page dedicated to quoting:

http://www.gnu.org/s/parallel/man.html#QUOTING

It even mentions the very error messages you write in your question.

If you can write it better please email your version to: [email protected].

like image 26
Ole Tange Avatar answered Nov 14 '22 23:11

Ole Tange