Python HTTP Server/Client: Remote end closed connection without response error

Question

I made simple HTTP Server using BaseHTTPRequestHandler. The problem is, that when I want to post some data using requests from client, I get ConnectionError. I did simple request from requests lib documentation. Also interesting thing is, that HTTP Server will receive data from client and print it to console. I don't understand how its possible.

Client:

def post_data():
    """Client method"""
    json_data = {
        'sender': 'User',
        'receiver': 'MY_SERVER',
        'message': 'Hello server! Sending some data.'}
    data_headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
    data_payload = json.dumps(json_data)

    try:
        post = requests.post('http://localhost:8080/post', data=data_payload,
                             headers=data_headers)
        print(post.status_code)
    except ConnectionError as e:
        print("CONNECTION ERROR: ")
        print(e)

HTTP Server:

def do_POST(self):
    """Server method"""
    self.send_response(200)
    print("Receiving new data ...")
    content_length = int(self.headers['Content-Length'])
    post_data = self.rfile.read(content_length)
    print(post_data)

Server result:

C:\Users\mypc\Projects\PythonFiles\httpserver>python server.py

Fri Jan  5 01:09:12 2018: HTTP Server started on port 8080.
127.0.0.1 - - [05/Jan/2018 01:09:21] "POST /post HTTP/1.1" 200 -
Receiving new data ...
b'{"sender": "User", "receiver": "MY_SERVER", "message": "Hello server! Sending some data."}'

Client result:

C:\Users\mypc\Projects\PythonFiles\httpserver>python client.py
CONNECTION ERROR:
('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

C:\Users\mypc\Projects\PythonFiles\httpserver>

Error without exception block:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Python36\lib\http\client.py", line 1331, in getresponse
    response.begin()
  File "C:\Python36\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "C:\Python36\lib\http\client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\requests\adapters.py", line 440, in send
    timeout=timeout
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "C:\Python36\lib\site-packages\urllib3\util\retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Python36\lib\site-packages\urllib3\packages\six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "C:\Python36\lib\site-packages\urllib3\connectionpool.py", line 383, in _make_request
    httplib_response = conn.getresponse()
  File "C:\Python36\lib\http\client.py", line 1331, in getresponse
    response.begin()
  File "C:\Python36\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "C:\Python36\lib\http\client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "client.py", line 137, in <module>
    start_parser()
  File "client.py", line 101, in start_parser
    send_requests(args.get, args.post)
  File "client.py", line 51, in send_requests
    post_data()
  File "client.py", line 129, in post_data
    headers=data_headers)
  File "C:\Python36\lib\site-packages\requests\api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "C:\Python36\lib\site-packages\requests\api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python36\lib\site-packages\requests\sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python36\lib\site-packages\requests\sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python36\lib\site-packages\requests\adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

Answer 1

It looks like the server is terminating the connection early without sending a full response. I've skimmed the docs and I think this is the problem (emphasis added):

send_response(code, message=None) 

Adds a response header to the headers buffer and logs the accepted request. The HTTP response line is written to the internal buffer, followed by Server and Date headers. The values for these two headers are picked up from the version_string() and date_time_string() methods, respectively. If the server does not intend to send any other headers using the send_header() method, then send_response() should be followed by an end_headers() call.

Changed in version 3.3: Headers are stored to an internal buffer and end_headers() needs to be called explicitly.

So you probably just need to add the call to end_headers. If you were reading an old example (prior to Python 3.3) this wouldn't have been needed.

Answer 2

You can just put end_headers as Peter Gibson said

self.send_header('Content-Type', 'blabla' )

self.end_headers()