Redirecting pv output to file

Question

I'm using the pv utility to asses the speed of log entries being added in a log file, like this:

tail -f -n 0 mylog.log | pv -lr -t -i 5 > /dev/null

This outputs the addition speed and refreshes every 5 sec:

0:00:05 [  10/s]

Now, I would like to redirect the current value ("10") to a file and refresh it every 5s. Not append it, just overwrite the previous value in the file. Any idea on how to do the redirection to file? I'm pretty new in unix and haven't got it to work yet.

Thanks!

Answer 1

pv is writing its diagnostics to stderr, so you can start with:

tail -f -n 0 mylog.log | pv -lr -t -i 5 > /dev/null 2> output-file

This will merely append, so output-file will end up with all of the data with lines terminated by carriage returns without linefeeds. (I don't have access to pv at the moment, so this is all conjecture.) To get the data you want, try some simple post processing along the lines of:

tail -f mylog.log | pv -lr -t -i 5 2>&1 > /dev/null | 
  tr /\\r \ \\n | tr -d [] | while read stamp value;
  do echo $value > output-file; done

The shell technique above allows you to direct stderr into the pipe for post-processing. Using tail like you're doing is a little odd, and pv is overkill for this. If you have such a long delay (5 seconds is a long time), it's probably acceptable to just use wc on each iteration and do subtraction:

echo 0 > output-file
while sleep 5; do
    expr $(wc -l < mylog.log) - $(cat output-file) > output-file
done

Note that this prints the difference over 5 seconds rather than a rate per second. expr does not handle non-integer arithmetic very well, so dividing by 5 is problematic. You could use bc or dc to get more precision, but you're probably better off not even doing subtraction and just writing the total number of lines to the file every few seconds.