How to add date string to each line of a continuously written log file

Question

Having a long running program that continuously writes to a logfile - how is it possible, disregarding any buffering issues, to add a date string to each line written to that file using a linux script?

I would imagine something like this:

tail -f logfile | ADD_DATE_TO_EACH_LINE > logfile2

The input would be something like that:

abc
def
ghi
jkl

The output should be similar to that:

2011-06-16 18:30:59 abc
2011-06-16 18:31:00 def
2011-06-16 18:35:21 ghi
2011-06-16 18:40:15 jkl

Answer 1

A little lengthy, but here is what I came up with:

tail -f logfile | sed -u 's/%/%%/g' | xargs -I {} date +"%Y-%m-%d %H:%M:%S {}"

Answer 2

A tool exists for that exact purpose, it’s ts (see man ts)

For example, with your logfile:

tail -f logfile | ts '%Y-%m-%d %H:%M:%S'

Also works with any software writing on the standard output, of course:

./blabla | ts '%Y-%m-%d %H:%M:%S'

If needed, you can add sub-second precision, with %.S instead of %S

Answer 3

You can try this

cat /etc/motd | xargs -d"\n" -I {} date +"%Y-%m-%d %H:%M:%S {}"

Example output:

2013-02-26 15:13:57 
2013-02-26 15:13:57 The programs included with the Debian GNU/Linux system are free software;
2013-02-26 15:13:57 the exact distribution terms for each program are described in the
2013-02-26 15:13:57 individual files in /usr/share/doc/*/copyright.
2013-02-26 15:13:57 
2013-02-26 15:13:57 Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
2013-02-26 15:13:57 permitted by applicable law.

Answer 4

With perl:

command 2>&1 | perl -pe 'print scalar(localtime()), " ";'

With gawk:

command 2>&1 | awk '{ print strftime(), $0; fflush() }'

Replace command with tail -f logfile for your specific example. Or, perhaps you could just redirect the original program's stdout/stderr to the above pipe.

Answer 5

Try

tail -f logfile | while read line; do echo `date` "$line" ; done