python releases output to tee very slowly [duplicate]

Question

I'm running a command like this:

python mycode.py | tee foobar.log 

This code does a lot of "print" to stdout, and without the pipe I can see some output immediately. With the pipe and tee, however, it takes long time until I see the first output, and then there is a lot of it. Looks like some buffer is waiting to be filled, and only when it is full, it dumps everything at once. I'm not sure whether it's tee, python or ubuntu problem. Same problems occurs with ipython for instance. What to do?

Thanks!

Answer 1

python -u mycode.py | tee foobar.log

-u means unbuffered standard input/output. Do check the man page, though. It has one slight catch in Python2 (also puts standard I/O into binary mode, which makes no difference in Ubuntu) and a different slight catch in Python3 (text-level I/O is still line buffered, so if your output doesn't contain many linebreaks you can still see significant buffering).