Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Read from two files in simultaneously in bash when they may be missing trailing newlines

Tags:

linux

bash

I have two text files that I'm trying to read through line by line at the same time. The files do not necessarily have the same number of lines, and the script should stop reading when it reaches the end of either file. I'd like to keep this as "pure" bash as possible. Most of the solutions I've found for doing this suggest something of the form:

while read -r f1 && read -r f2 <&3; do
    echo "$f1"
    echo "$f2"
done < file1 3<file2

However this fails if the last line of a file does not have a newline.

If I were only reading one file I would do something like:

while IFS='' read -r line || [[ -n "$line" ]]; do
    echo "$line"
done < file1

but when I try to extend this to reading multiple files by doing things like:

while IFS='' read -r f1 || [[ -n "$f1" ]] && read -r f2 <&3 || [[ -n "$f2" ]]; do
    echo "$f1"
    echo "$f2"
done < file1 3<file2

or

while IFS='' read -r f1 && read -r f2 <&3 || [[ -n "$f1" || -n "$f2" ]]; do
    echo "$f1"
    echo "$f2"
done < file1 3<file2

I can't seem to get the logic right and the loop either doesn't terminate or finishes without reading the last line.

I can get the desired behavior using:

while true; do
    read -r f1 <&3 || if [[ -z "$f1" ]]; then break; fi;
    read -r f2 <&4 || if [[ -z "$f2" ]]; then break; fi;
    echo "$f1"
    echo "$f2"
done 3<file1 4<file2

However this doesn't seem to match the normal (?)

while ... read ...; do
     ... 
done

idiom that I see for reading files.

Is there a better way to simultaneously read from two files that might have differing numbers of lines and last lines that are not newline terminated?

What are the potential drawbacks of my way of reading the files?

like image 947
shay Avatar asked Jan 17 '26 10:01

shay


1 Answers

You can override the precedence of the && and || operators by grouping them with { }:

while { IFS= read -r f1 || [[ -n "$f1" ]]; } && { IFS= read -r f2 <&3 || [[ -n "$f2" ]]; }; do

Some notes: You can't use ( ) for grouping in this case because that forces its contents to run in a subshell, and variables read in subshells aren't available in the parent shell. Also, you need a ; before each }, so the shell can tell it isn't just a parameter to the last command. Finally, you need IFS= (or equivalently IFS='') before each read command, since assignments given as a prefix to a command apply only to that one command.

like image 87
Gordon Davisson Avatar answered Jan 21 '26 07:01

Gordon Davisson



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!