Jenkins: how to see progress in the log output, when it gets constantly overwritten?

Question

I'm running long (few hours) Rsync backup tasks in Jenkins.
Rsync prints progress output to log. While looking in terminal - last line is "refreshing", i.e. it constantly prints over itself. But when Jenkins runs the task - it doesn't show that info.

Here is what I see in Jenkins, while the task is running:

And here is what I see, after it's completed (and that's what I want to see live, while it running):

sending incremental file list
35-openMeet-flat.vmdk

        131,072   0%    0.00kB/s    0:00:00  
      9,437,184   0%    8.88MB/s    1:38:26  
     21,757,952   0%   10.30MB/s    1:24:49  
     32,899,072   0%   10.40MB/s    1:23:58  
     44,302,336   0%   10.49MB/s    1:23:12  
     55,443,456   0%   10.92MB/s    1:19:55  
     66,191,360   0%   10.56MB/s    1:22:40  
     78,118,912   0%   10.73MB/s    1:21:17  

How can I configure Jenkins to print complete output while the task is running?

P.S. I would be happy, even if I would need to find and watch some Jenkins log file. But currently I can't find anything. For example I tried this:

slavik@ubhome:/var/lib/jenkins/jobs/backup ESXI VM/builds/34$ tail log
sending incremental file list
35-openMeet.vmdk
            532 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 434 bytes  received 41 bytes  190.00 bytes/sec
total size is 532  speedup is 1.12
Comparing file sizes...
Sizes are different, calculating delta checksums, can take a while, time for a coffee...
sending incremental file list
35-openMeet-flat.vmdk
slavik@ubhome:/var/lib/jenkins/jobs/backup ESXI VM/builds/34$

But as you can see, it doesn't show last lines with progress data. Looks like it holds is in RAM and doesn't write to disk until it is really new line.

Answer 1

Sure this is over a year later but I stumbled upon this question in hopes of finding the answer. Failing that I managed to find a solution and thought I'd share for the rest of the internet.

What you want to do is "unbuffer" the rsync command. There are a couple projects out there that can "unbuffer" things (unbuffered, expect's unbuffer command) but these didn't work quite right for me. Unbuffered was close but threw a bunch of newlines into my log that I didn't want (after using the -c flag which prevented the duplicate lines everywhere).

What ended up working was this: piping my rsync command through stbuf with the flags found at the link. Now my jenkins console is nice and pretty and keeps up with rsync's progress:

rsync -avz --progress <source> <destination> | stdbuf -oL tr '\r' '\n'