http.server not responding until after the connection closes

Question

I have an application which uses the requests package to POST to a server. In my testing environment, I want to stand up a simple server so that I can validate the client's requests. However, when I call requests.post, it just hangs. The server doesn't respond at all until after the client finally times out and the connection closes. Ironically, the server then sends a 200 OK.

Here's my MRE for the server:

import http.server

class TestHandler(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        print('Received:', self.rfile.read())
        self.send_response(200)
        self.end_headers()

if __name__ == '__main__':
    http.server.HTTPServer(('127.0.0.1', 8080), TestHandler).serve_forever()

Here's my MRE for the client:

import requests

if __name__ == '__main__':
    print(requests.post('http://127.0.0.1:8080/hello', data='Testing', timeout=5))

Answer 1

self.rfile.read() reads the socket until it closes, which is why when the client times out and closes the socket you finally send the response on the server. You need to provide a length, which can be found in the headers. Try this server:

import http.server

class TestHandler(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        print(self.headers)
        length = int(self.headers['content-length'])
        print('Received:', self.rfile.read(length))
        self.send_response(200)
        self.end_headers()

if __name__ == '__main__':
    http.server.HTTPServer(('127.0.0.1', 8080), TestHandler).serve_forever()

Output with your client:

Host: 127.0.0.1:8080
User-Agent: python-requests/2.25.1
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Length: 7


Received: b'Testing'
127.0.0.1 - - [10/Dec/2021 09:50:39] "POST /hello HTTP/1.1" 200 -