Python app does not print anything when running detached in docker

Question

Finally I found a solution to see Python output when running daemonized in Docker, thanks to @ahmetalpbalkan over at GitHub. Answering it here myself for further reference :

Using unbuffered output with

CMD ["python","-u","main.py"]

instead of

CMD ["python","main.py"]

solves the problem; you can see the output (both, stderr and stdout) via

docker logs myapp

now!

In my case, running Python with -u didn't change anything. What did the trick, however, was to set PYTHONUNBUFFERED=1 as environment variable:

docker run --name=myapp -e PYTHONUNBUFFERED=1 -d myappimage

[Edit]: Updated PYTHONUNBUFFERED=0 to PYTHONUNBUFFERED=1 after Lars's comment. This doesn't change the behavior and adds clarity.

See this article which explain detail reason for the behavior:

There are typically three modes for buffering:

• If a file descriptor is unbuffered then no buffering occurs whatsoever, and function calls that read or write data occur immediately (and will block).
• If a file descriptor is fully-buffered then a fixed-size buffer is used, and read or write calls simply read or write from the buffer. The buffer isn’t flushed until it fills up.
• If a file descriptor is line-buffered then the buffering waits until it sees a newline character. So data will buffer and buffer until a \n is seen, and then all of the data that buffered is flushed at that point in time. In reality there’s typically a maximum size on the buffer (just as in the fully-buffered case), so the rule is actually more like “buffer until a newline character is seen or 4096 bytes of data are encountered, whichever occurs first”.

And GNU libc (glibc) uses the following rules for buffering:

Stream               Type          Behavior
stdin                input         line-buffered
stdout (TTY)         output        line-buffered
stdout (not a TTY)   output        fully-buffered
stderr               output        unbuffered

So, if use -t, from docker document, it will allocate a pseudo-tty, then stdout becomes line-buffered, thus docker run --name=myapp -it myappimage could see the one-line output.

And, if just use -d, no tty was allocated, then, stdout is fully-buffered, one line App started surely not able to flush the buffer.

Then, use -dt to make stdout line buffered or add -u in python to flush the buffer is the way to fix it.

If you want to add your print output to your Flask output when running docker-compose up, add the following to your docker compose file.

web:
  environment:
    - PYTHONUNBUFFERED=1

https://docs.docker.com/compose/environment-variables/

Since I haven't seen this answer yet:

You can also flush stdout after you print to it:

import time

if __name__ == '__main__':
    while True:
        print('cleaner is up', flush=True)
        time.sleep(5)