Reading from asyncio StreamReader

Question

I am trying to extend the python asyncio HTTP server example that uses a streaming reader/writer (code). If I understand it correctly, the example handler read 100 bytes from the reader and echoes it back to the client through the writer. I am trying to read more than 100 bytes... reading until there is nothing more to read would be nice.

I have tried letting the read() function read as much as possible,

data = yield from reader.read()

but that seems to block forever. So I tried reading chunks until the EOF is reached,

while not reader.at_eof():
    data += yield from reader.read(100)

and while this retrieves more of the data, it to seems to block on the read call instead of exiting the while loop.

How can I get the entire message from the client using the stream reader?

Answer 1

You should check if StreamReader.read returned an empty bytes object to signal an EOF:

data = bytearray()
while True:
    chunk = yield from reader.read(100)
    if not chunk:
        break
    data += chunk

Also, consider using aiohttp if you need a fully functional HTTP client.

Answer 2

Like so:

empty_bytes = b''
result = empty_bytes

while True:
    chunk = await response.content.read(8)

    if chunk == empty_bytes:
        break

    result += chunk

To determine the EOF use

if chunk == empty_bytes:

in stead of

if not chunk:

See the docs (aiohttp): the read returns an empty byte string

b''

on EOF, so check for that explicitly.

Note: If you would like to read until the end of the chunk as it was delivered from the server, checkout

StreamReader.readchunk()

(Didn't test it, though.)