Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

xargs split at newlines not spaces

Tags:

shell

unix

xargs

here is my problem in short

$ echo 'for i in $@; do echo arg: $i; done; echo DONE' > /tmp/test.sh $ echo "ac\nbc\ncc\n" | xargs bash /tmp/test.sh  arg: ac arg: bc arg: cc DONE 

Which is what i expect, but

$ echo "ac s\nbc s\ncc s\n" | xargs -d \n bash /tmp/test.sh arg: ac arg: s arg: bc arg: s arg: cc arg: s DONE 

Shouldn't the output be?

arg: ac s arg: bc s arg: cc s DONE 

How do I get the 2nd output with xargs?

like image 677
fakedrake Avatar asked Apr 17 '14 20:04

fakedrake


People also ask

Does xargs run in parallel?

xargs will run the first two commands in parallel, and then whenever one of them terminates, it will start another one, until the entire job is done. The same idea can be generalized to as many processors as you have handy. It also generalizes to other resources besides processors.

What is the point of xargs?

Overview. xargs (short for “eXtended ARGuments”) is a command on Unix and most Unix-like operating systems. It converts input from standard input (STDIN) into arguments to a command.

What does piping to xargs do?

The xargs utility allows you to build and execute commands from standard input. It is usually used in combination with other commands through piping. With xargs , you can provide standard input as argument to command-line utilities like mkdir and rm .

How do you pass arguments to xargs?

The -c flag to sh only accepts one argument while xargs is splitting the arguments on whitespace - that's why the double quoting works (one level to make it a single word for the shell, one for xargs).


1 Answers

Try:

printf %b 'ac s\nbc s\ncc s\n' | xargs -d '\n' bash /tmp/test.sh 

You neglected to quote the \n passed to -d, which means that just n rather than \n was passed to xargs as the delimiter - the shell "ate" the \ (when the shell parses an unquoted string, \ functions as an escape character; if an ordinary character follows the \ - n in this case - only that ordinary character is used).

Also heed @glenn jackman's advice to double-quote the $@ inside the script (or omit the in "$@" part altogether).

Also: xargs -d is a GNU extension, which, for instance, won't work on FreeBSD/macOS. To make it work there, see @glenn jackman's xargs -0-based solution.


Note that I'm using printf rather than echo to ensure that the \n instances in the string are interpreted as newlines in all Bourne-like shells:
In bash and ksh[1], echo defaults to NOT interpreting \-based escape sequences (you have to use -e to achieve that) - unlike in zsh and strictly POSIX-compliant shells such as dash.
Therefore, printf is the more portable choice.

[1] According to the manual, ksh's echo builtin exhibits the same behavior as the host platform's external echo utility; while this may vary across platforms, the Linux and BSD/macOS implementations do not interpret \ escape sequences by default.

like image 80
mklement0 Avatar answered Oct 04 '22 08:10

mklement0