Non-blocking file read

Question

How can I read the contents of a binary or a text file in a non-blocking mode?

For binary files: when I open(filename, mode='rb'), I get an instance of io.BufferedReader. The documentation fort io.BufferedReader.read says:

Read and return size bytes, or if size is not given or negative, until EOF or if the read call would block in non-blocking mode.

Obviously a straightforward open(filename, 'rb').read() is in a blocking mode. To my surprise, I could not find an explanation anywhere in the io docs of how to choose the non-blocking mode.

For text files: when I open(filename, mode='rt'), I get io.TextIOWrapper. I assume the relevant docs are those for read in its base class, io.TextIOBase; and according to those docs, there seems no way to do non-blocking read at all:

Read and return at most size characters from the stream as a single str. If size is negative or None, reads until EOF.

Answer 1

Python does support non-blocking reads, at least on Unix type systems, by setting the O_NONBLOCK flag. In Python 3.5+, there is the os.set_blocking() function which makes this easier:

import os
f = open(filename, 'rb')
os.set_blocking(f.fileno(), False)
f.read()  # This will be non-blocking.

However, as zvone's answer notes, this doesn't necessarily work on actual disk files. This isn't a Python thing though, but an OS limitation. As the Linux open(2) man page states:

Note that this flag has no effect for regular files and block devices; that is, I/O operations will (briefly) block when device activity is required, regardless of whether O_NONBLOCK is set.

But it does suggest this may be implemented in the future:

Since O_NONBLOCK semantics might eventually be implemented, applications should not depend upon blocking behavior when specifying this flag for regular files and block devices.

Answer 2

I suggest using aiofiles - a library for handling local disk files in asyncio applications.

import aiofiles

async def read_without_blocking():
    f = await aiofiles.open('filename', mode='r')
    try:
        contents = await f.read()
    finally:
        await f.close()