How to run grep inside awk?

Question

Suppose I have a file input.txt with few columns and few rows, the first column is the key, and a directory dir with files which contain some of these keys. I want to find all lines in the files in dir which contain these key words. At first I tried to run the command

cat input.txt | awk '{print $1}' | xargs grep dir

This doesn't work because it thinks the keys are paths on my file system. Next I tried something like

cat input.txt | awk '{system("grep -rn dir $1")}'

But this didn't work either, eventually I have to admit that even this doesn't work

cat input.txt | awk '{system("echo $1")}'

After I tried to use \ to escape the white space and the $ sign, I came here to ask for your advice, any ideas?

Of course I can do something like

for x in `cat input.txt` ; do grep -rn $x dir ; done

This is not good enough, because it takes two commands, but I want only one. This also shows why xargs doesn't work, the parameter is not the last argument

Answer 1

You don't need grep with awk, and you don't need cat to open files:

awk 'NR==FNR{keys[$1]; next} {for (key in keys) if ($0 ~ key) {print FILENAME, $0; next} }' input.txt dir/*

Nor do you need xargs, or shell loops or anything else - just one simple awk command does it all.

If input.txt is not a file, then tweak the above to:

real_input_generating_command |
awk 'NR==FNR{keys[$1]; next} {for (key in keys) if ($0 ~ key) {print FILENAME, $0; next} }' - dir/*

All it's doing is creating an array of keys from the first file (or input stream) and then looking for each key from that array in every file in the dir directory.

Answer 2

Try following

awk '{print $1}' input.txt | xargs -n 1 -I pattern grep -rn pattern dir

Answer 3

First thing you should do is research this.

Next ... you don't need to grep inside awk. That's completely redundant. It's like ... stuffing your turkey with .. a turkey.

Awk can process input and do "grep" like things itself, without the need to launch the grep command. But you don't even need to do this. Adapting your first example:

awk '{print $1}' input.txt | xargs -n 1 -I % grep % dir

This uses xargs' -I option to put xargs' input into a different place on the command line it runs. In FreeBSD or OSX, you would use a -J option instead.

But I prefer your for loop idea, converted into a while loop:

while read key junk; do grep -rn "$key" dir ; done < input.txt