What does it mean to flush file contents in Python?

Question

I am trying to teach myself Python by reading documentation. I am trying to understand what it means to flush a file buffer. According to documentation, "file.flush" does the following.

Flush the internal buffer, like stdio‘s fflush().
This may be a no-op on some file-like objects.

I don't know what "internal buffer" and "no-op" mean, but I think it says that flush writes data from some buffer to a file.

Hence, I ran this file toggling the pound sign in the line in the middle.

with open("myFile.txt", "w+") as file:
    file.write("foo")
    file.write("bar")
    # file.flush()
    file.write("baz")
    file.write("quux")

However, I seem to get the same myFile.txt with and without the call to file.flush(). What effect does file.flush() have?

Answer 1

Python buffers writes to files. That is, file.write returns before the data is actually written to your hard drive. The main motivation of this is that a few large writes are much faster than many tiny writes, so by saving up the output of file.write until a bit has accumulated, Python can maintain good writing speeds.

file.flush forces the data to be written out at that moment. This is hand when you know that it might be a while before you have more data to write out, but you want other processes to be able to view the data you've already written. Imagine a log file that grows slowly. You don't want to have to wait ages before enough entries have built up to cause the data to be written out in one big chunk.

In either case, file.close causes the remaining data to be flushed, so "quux" in your code will be written out as soon as file (which is a really bad name as it shadows the builtin file constructor) falls out of scope of the with context manager and gets closed.

Note: your OS does some buffering of its own, but I believe every OS where Python is implemented will honor file.flush's request to write data out to the drive. Someone please correct me if I'm wrong.

By the way, "no-op" means "no operation", as in it won't actually do anything. For example, StringIO objects manipulate strings in memory, not files on your hard drive. StringIO.flush probably just immediately returns because there's not really anything for it to do.

Answer 2

Buffer content might be cached to improve performance. Flush makes sure that the content is written to disk completely, avoiding data loss. It is also useful when, for example, you want the line asking for user input printed completely on-screen before the next file operation takes place.