Why is python-requests to localhost slow?

Question

I have a simple flask server defined like so:

import sys

import flask
from flask import request

app = flask.Flask(__name__)
port = 4057

@app.route('/search', methods=['POST'])
def search():
  request.json['query']
  results = ['fake', 'data']
  return flask.jsonify(results)

if __name__ == '__main__':
 app.config['TEMPLATES_AUTO_RELOAD'] = True
 app.run(host='0.0.0.0', port=port, debug=(port != 80))

I have a simple client defined like this:

import json

import requests

headers = {'content-type': 'application/json'}
resp = requests.post('http://localhost:4057/search', json.dumps({'query': 'foo'}), headers=headers)
print resp.content

The client works, but it takes like 3 seconds to complete the request.

curl completes in like half a second:

curl 'http://localhost:4057/search' -H 'Content-Type: application/json' -d '{"query": "foo"}'

Answer 1

Try 127.0.0.1 There maybe some odd name resolution rules fumbling requests.

Ah ok, this is my work machine. I took a look at /etc/hosts and saw ~200 routes defined that I didn't realize were there

As mention in the comments, this does not explain the odd behavior replicated using curl.

Answer 2

I've recently encountered a similar issue with slow connections from requests to 'localhost', and also found that using '127.0.0.1' was much faster.

After some investigation, I found that in my case, the slow connections on Windows were related to urllib3 (the transport library used by requests) attempting an IPv6 connection first, which is refused since the server is listening on IPv4 only. For Windows only, however, there is an unavoidable 1 sec timeout on socket.connect for refused TCP connections, while the OS 'helpfully' retries the connection up to 3 times. (See: Why do failed attempts of Socket.connect take 1 sec on Windows?).

On Linux, socket.connect fails immediately if the connection is refused, and it can attempt the IPv4 connection without any delay. In addition, some software, like curl, support limiting name resolution to only IPv4 or IPv6. Unfortunately, requests and urllib3 do not support this, and do not seem to plan to support this anytime soon (See: https://github.com/psf/requests/issues/1691)