awk command inside while loop creates empty output files

Question

I am trying to loop through the lines of a file (Names.txt) and looking for the expressions of each line on another file (Data.txt). I want to print the lines of Data.txt containing one expression on a file line.txt. Say my file Names.txt looks like this:

A
B
C
D

And my file Data.txt looks like this:

A    info1
D    info2
C    info3
B    info4
E    info5
F    info6

I used the following command directly on Terminal (Mac OSX):

while read line; do awk '/$line/ { print $0 }' Data.txt > $line.txt; done < Names.txt

Though all the files $line.txt are created, they are unfortunately all empty. It seems to be an error with my use of the while loop, since when I only put the awk command to look for a single expression it works just fine...

Thanks for any help!

Answer 1

Replace:

awk '/$line/ { print $0 }' Data.txt

With:

awk -v line="$line" '$0 ~ line { print $0 }' Data.txt

The problem is that line is a shell variable. The top line has $line inside of single-quotes. The shell won't touch anything inside single-quotes. So, the print statement is only executed if the line matches a literal dollar sign followed by l, i, n, and e.

In the second version above, the -v option is used to create an awk variable called line. The awk variable can then be used in the script.

Another approach, possibly dangerous

We could try to use the shell variable line directly by placing it in double-quotes:

awk "/$line/"' { print $0 }' Data.txt

Don't do this unless you have trust in the contents of line. For example, if line had the following value:

line='./ {print "Ha Ha"} /A'

Then, the awk script would execute without error but it would do some unexpected things. Because this can be a security issue, it is much safer to use the awk variable approach described above.

A Somewhat Cryptic Simplification

If an awk command has a condition but not an action, the default is print $0. This means that, if you like, the awk command can be simplified to:

awk -v line="$line" '$0 ~ line' Data.txt

A Suggestion

It looks like you want to match the lines in Names.txt with the first column of Data.txt. If that is the case, you may want to refine the condition to avoid false matches by using:

awk -v line="$line" '$1 ~ line' Data.txt