Why is my Python script not writing to file when it is backgrounded it in Linux?

Question

I can do:

python script.py > logfile 2>&1

and once I kill it using ctrl c I can see changes to the logfile. However, when I do:

python script.py > logfile 2>&1 &

I can't see any changes to the log file . How can I background my script and still have it write to the logfile?

Answer 1

Your writes aren't going into the ether -- they're getting buffered. After a graceful shutdown (SIGTERM, not SIGKILL), you should see content in the file then as well.

Alternately, you can explicitly flush:

sys.stdout.flush()

...or tell Python to run in unbuffered mode:

python -u yourscript.py

or start your script with

#!/usr/bin/python -u

...to ensure that content is written. This will, of course, have a performance penalty, since your script will be trying to perform IO more frequently.

---

(As an aside, #!/usr/bin/env python -u is not guaranteed to work: Only a command name and a single argument are guaranteed to be passed through from a shebang line to a command that's run; your operating system may or may not honor more than that).