Read stdin as binary using coroutines

Question

I have used aiofiles to read files if we have path to the file

#! /usr/bin/python3.6
import asyncio
import aiofiles

async def read_input():
    async with aiofiles.open('hello.txt', mode='rb') as f:
        while True:
            inp = await f.read(4)
            if not inp: break
            print('got :' , inp)

async def main():
    await asyncio.wait([read_input()])

event_loop = asyncio.get_event_loop()
event_loop.run_until_complete(main())
event_loop.run_forever()

I need to read the file from standard input as ./reader.py < file.txt The above code reads file as binary. But I need to read stdin as binary using coroutines. I am unable to figure out any way to do that.

Answer 1

I am posting possible solutions to my own question.

the first soln is as suggested by @user4815162342 :

async def read_input():
    async with aiofiles.open('/dev/stdin', mode='rb') as f:
        while True:
            inp = await f.read(4)
            if not inp: break
            print('got :' , inp)

it is platform specific doesn't work on windows.

Another possible solution to this problem is using an feature of aioconsole:

async def read_input():
    stdin, _ = await aioconsole.get_standard_streams()
    while True:
        line = await stdin.read(4)
        if not line: break
        print('got',line)

this seems more robust and platform independent. There is a small problem with this too, it work fine for normal file while it cannot read a compressed file like ./check.py < hello.bz2. On the other hand the aiofile approach works fine with ./check.py < hello.bz2.

NOTE: on execution on ./check.py < hello.bz2 with aioconsole approach we get this error

Task exception was never retrieved
future: <Task finished coro=<read_aioconsole() done, defined at ./input.py:21> exception=UnicodeDecodeError('utf-8', b'BZh91AY&SY\x89\x9a\xef"\x00\x00\x05\xc8\x00\x00\x10\x1f\x80 \x001\x0c\x00\xd0\xf4\'\r\xd0\xb5%\xe2\xeeH\xa7\n\x12\x113]\xe4@', 10, 11, 'invalid start byte')>
Traceback (most recent call last):
  File "./input.py", line 24, in read_aioconsole
    line = await stdin.read(4)
  File "/usr/local/lib/python3.6/dist-packages/aioconsole/stream.py", line 82, in read
    data = yield from self.loop.run_in_executor(None, self.stream.read, n)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 10: invalid start byte