Bash: Looping over multiple input sources (nesting "while read" loops?)

Question

I have a query.sh script which runs dig commands, performing a set of lookups from different DNS servers, which are currently given in a column in the single input file used (sites.txt).

My goal is to modify this script to use a different input file, dns_servers.txt, to find the DNS servers to iterate through for each query.

I'm not clear on where to start here. What do I need to do to be able to safely nest while read loops?

---

Current inputs:

query.sh

#!/bin/sh

while read line;
do
        set $line
        echo "SITE:" $1@$2
        /usr/sbin/dig +short -4 @$2 $1
        sleep 5
        exitStatus=$?
        echo "Exit Status: " $exitStatus
done < sites.txt

sites.txt

Current format has a hostname and a DNS server to use for lookups against that hostname.

www.google.com 8.8.4.4

The intent is for the column with the DNS server to be ignored, and the contents of dns_servers.txt to be used instead.

---

Desired Inputs

dns_servers.txt

10.1.1.1
12.104.1.232
...

Answer 1

Ignoring any additional column(s) in the sites.txt file, and iterating through the lines of dns_servers.txt, might look like the following:

#!/bin/sh
while read -r site _ <&3; do
  while read -r dns_server <&4; do
    dig +short -4 "@$dns_server" "$site"; exit=$?
    sleep 5
    echo "Exit status: $exit"
  done 4<dns_servers.txt
done 3<sites.txt

The key changes here:

• Pass the list of fields you want to parse as arguments to read. The underscore passed as second positional argument to the first read is the variable name to which the second column of site.txt is now being saved.
• Nest your loops, since you want to read from the inner loop for every pass of the outer loop.
• Use a different file descriptor (here, 3 and 4) for outer and inner loops to keep them separated.

---

Incidentally, if you were targeting bash (#!/bin/bash) rather than POSIX sh (#!/bin/sh), I might do this differently. The below uses the bash 4 extension mapfile to read dns_servers.txt all at once:

#!/bin/bash
readarray -t dns_servers <dns_servers.txt
while read -r site _; do
  for from_ip in "${dns_servers[@]}"; do
    dig +short -4 "@$from_ip" "$site"; exit=$?
    sleep 5
    echo "Exit status: $exit"
  done
done <sites.txt

Here, we read dns_servers.txt only once, and reuse that list of values for each value read from sites.txt.

If you're using bash 3.x, mapfile can be replaced with a loop:

dns_servers=( )
while read -r; do dns_servers+=( "$REPLY" ); done <dns_servers.txt