Tail the most recent file

Question

I have a program that logs its output to an *.out file when running. I set up a bash script to run several different times, so each run writes to a different *.out file. I found that I could tail the most recent *.out file like this:

watch tail $(ls -tr *.out | tail -n1)

The problem seems to be that the referencing $() only gets executed once. So when the first run finishes, watch continues to tail the same *.out file, even though there is now a newer *.out file.

How can I alter this to go to the next *.out file after one run finishes?

I tried doing some nesting of quotes and parentheses, but I don't really understand the details of referencing. Matters are complicated by the fact that watch passes its command to sh, even though I'm using bash.

Bonus points: it would be great if this could be modified to tail -f instead of just repeating watch every n seconds.

Answer 1

I also dealt with this problem and finally came to this solution:

 watch "ls -t1 | sed q | xargs tail"

Need to get a bit hacky for the bonus. The tail command also supports an argument --pid which makes tail die as soon as the referenced PID dies. Sounds good? Here's some pattern you could use:

while true; do ls -t1 | sed q | xargs lsof -F np | sed 's/.\(.*\)/\1/g' | xargs tail -f --pid 2>/dev/null ; done

This basically tracks the most current file and restarts tail when the file writing process ends.

Answer 2

If all the files exist to begin with, and you have a modern implementation of tail that can handle multiple files,

tail -f *.out

would seem to be the best solution. But if you don't have all the files to begin with, then it would still be easiest if you just created them. *.out will expand only to the files you already have.

But if you can't know the filenames for tail -f before you start, then you will need to restart tail if any new outfiles are created while you are tailing. In that case, given you are only writing to each outfile one at a time, you could do:

inotifywait -m -e create -q . | while read dir event file; do
    kill %tail
    tail -f "$file" &
done
kill %tail

(The above requires inotifywait, which is included in inotify-tools on Debian-based systems like Ubuntu.) It watches the current directory for newly created files, tails whichever file was created, in the background, and kills the previous tail command whenever a new file is created.