RuntimeError: No response returned in FastAPI when refresh request

Question

I got this error in my application and i didn't know why. After many search and debugging just figured out that it happens when i refresh my request before getting response(cancel request and send another request while processing previous request). Because of that my application need more than 2 seconds to respond, i get too many of this type of error.

So far i know its from my middleware but i don't know why it happens and what should i do.

Any idea how to fix this issue ?

This is the error i get:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 81, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 76, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 35, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 101, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 367, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 208, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 55, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/gateway/./app/core/middlewares.py", line 26, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 37, in call_next
    raise RuntimeError("No response returned.")
RuntimeError: No response returned.

and this is my middleware:

class LoggerMiddleWare(BaseHTTPMiddleware):

    def __init__(self, app: ASGIApp):
        super().__init__(app)
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.setLevel(logging.INFO)
        file_handler = logging.FileHandler('api.log')
        file_handler.setFormatter(JSONFormatter())
        self.logger.addHandler(file_handler)
        self.logger.addFilter(APIFilter())

    async def dispatch(self, request: Request, call_next):
        request.state.services = {}
        response = await call_next(request)
        self.logger.info(None, extra={'request': request, 'response': response})

        return response

I'm using fastapi 0.73 and starlette 0.17.1.

---

To reproduce this issue, we need to add two middlewares.

A minimal reproducible example can be found here: https://github.com/encode/starlette/issues/1634#issuecomment-1124806406

Answer 1

Update: As of 14 Nov 2022, this has been fixed in starlette==0.21.0 and fastapi==0.87.0.

---

This is due to how starlette uses anyio memory object streams with StreamingResponse in BaseHTTPMiddleware.

1. When you cancel a request, the ASGI app receives the "http.disconnect" message.
2. After your route function returns, your last middleware will await response(...).
3. StreamingResponse's async def __call__ will call self.listen_for_disconnect and then task_group.cancel_scope.cancel() since the request is already disconnected. The stream is closed by the cancellation check in await checkpoint() of MemoryObjectSendStream.send before it has a chance to send the "http.response.start" message.
4. Your second-to-last and earlier middlewares will encounter anyio.EndOfStream while await recv_stream.receive() in this part of BaseHTTPMiddleware's __call__ method:

   try:
       message = await recv_stream.receive()
   except anyio.EndOfStream:
       if app_exc is not None:
           raise app_exc
       raise RuntimeError("No response returned.")
   
   assert message["type"] == "http.response.start"
   

#4 is why, to reproduce this issue, we need two middlewares that inherit BaseHTTPMiddleware.

Workaround 1

You can subclass BaseHTTPMiddleware to ignore that exception if the request is disconnected:

class MyBaseHTTPMiddleware(BaseHTTPMiddleware):

    async def __call__(self, scope, receive, send):
        try:
            await super().__call__(scope, receive, send)
        except RuntimeError as exc:
            if str(exc) == 'No response returned.':
                request = Request(scope, receive=receive)
                if await request.is_disconnected():
                    return
            raise

    async def dispatch(self, request, call_next):
        raise NotImplementedError()

Usage:

# class LoggerMiddleWare(BaseHTTPMiddleware):
class LoggerMiddleWare(MyBaseHTTPMiddleware):

Workaround 2

Actually, only the outermost BaseHTTPMiddleware needs to handle the exception, so you can just implement a SuppressNoResponseReturnedMiddleware and put it as your first middleware:

class SuppressNoResponseReturnedMiddleware(BaseHTTPMiddleware):

    async def dispatch(self, request, call_next):
        try:
            return await call_next(request)
        except RuntimeError as exc:
            if str(exc) == 'No response returned.' and await request.is_disconnected():
                return Response(status_code=HTTP_204_NO_CONTENT)
            raise

Reference: https://github.com/encode/starlette/discussions/1527#discussioncomment-2234702