Unix Pipes for Command Argument

Question

I am looking for insight as to how pipes can be used to pass standard output as the arguments for other commands.

For example, consider this case:

ls | grep Hello

The structure of grep follows the pattern: grep SearchTerm PathOfFileToBeSearched. In the case I have illustrated, the word Hello is taken as the SearchTerm and the result of ls is used as the file to be searched. But what if I want to switch it around? What if I want the standard output of ls to be the SearchTerm, with the argument following grep being PathOfFileToBeSearched? In a general sense, I want to have control over which argument the pipe fills with the standard output of the previous command. Is this possible, or does it depend on how the script for the command (e.g., grep) was written?

Thank you so much for your help!

Answer 1

grep itself will be built such that if you've not specified a file name, it will open stdin (and thus get the output of ls). There's no real generic mechanism here - merely convention.

If you want the output of ls to be the search term, you can do this via the shell. Make use of a subshell and substitution thus:

$ grep $(ls) filename.txt

In this scenario ls is run in a subshell, and its stdout is captured and inserted in the command line as an argument for grep. Note that if the ls output contains spaces, this will cause confusion for grep.

Answer 2

There are basically two options for this: shell command substitution and xargs. Brian Agnew has just written about the former. xargs is a utility which takes its stdin and turns it into arguments of a command to execute. So you could run

ls | xargs -n1 -J % grep -- % PathOfFileToBeSearched

and it would, for each file output by ls, run grep -e filename PathOfFileToBeSearched to grep for the filename output by ls within the other file you specify. This is an unusual xargs invocation; usually it's used to add one or more arguments at the end of a command, while here it should add exactly one argument in a specific place, so I've used -n and -J arguments to arrange that. The more common usage would be something like

ls | xargs grep -- term

to search all of the files output by ls for term. Although of course if you just want files in the current directory, you can this more simply without a pipeline:

grep -- term *

and likewise in your reversed arrangement,

for filename in *; do
  grep -- "$@" PathOfFileToBeSearched
done

There's one important xargs caveat: whitespace characters in the filenames generated by ls won't be handled too well. To do that, provided you have GNU utilities, you can use find instead.

find . -mindepth 1 -maxdepth 1 -print0 | xargs -0 -n1 -J % grep -- % PathOfFileToBeSearched

to use NUL characters to separate filenames instead of whitespace