Python print statements being buffered with > output redirection

Question

I'm doing print statements in python. I'm executing my script like so:

python script.py > out.log nohup &

The print statements are not all showing up in out.log but the program is finishing ok.

That line of code is in an .sh file I execute by doing ./script.sh

Update: The log does get all the data but not until a certain # of lines are printed. It would appear to be buffering the output.

Answer 1

When stdout is sent to a tty it will be line buffered and will be flushed every line, but when redirected to a file or pipe it'll be fully buffered and will only be flushed periodically when you overrun the buffer.

You'll have to add sys.stdout.flush() calls after each line if you want the output to be immediately visible, or disable buffering entirely. See Disable output buffering for ways to do the latter:

1. Use the -u command line switch
2. Wrap sys.stdout in an object that flushes after every write
3. Set PYTHONUNBUFFERED env var
4. sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)