Python's 2.7 socket.timeout behavior

Question

Below are two simple python functions. First tries to connect to test.com domain on 666 (host name is valid but port is not). Second tries to connect to imap.mail.outlook.com on port 993 (host name is valid, but looks like not for public use/access).

def fn_outlook(timeout):
    try:
        socket.create_connection(("imap.mail.outlook.com", 993), timeout=timeout)
    except socket.timeout:
        pass


def fn_test(timeout):
    try:
        socket.create_connection(("test.com", 666), timeout=timeout)
    except socket.timeout:
        pass

And here are execution time for that function with different timeouts:

In [14]: %time fn_test(1)
CPU times: user 644 µs, sys: 1.07 ms, total: 1.71 ms
Wall time: 1 s

In [15]: %time fn_test(2)
CPU times: user 589 µs, sys: 1.15 ms, total: 1.74 ms
Wall time: 2 s

In [16]: %time fn_outlook(2)
CPU times: user 838 µs, sys: 2.24 ms, total: 3.08 ms
Wall time: 7.15 s

In [17]: %time fn_outlook(4)
CPU times: user 705 µs, sys: 1.18 ms, total: 1.88 ms
Wall time: 12 s

In [18]: %time fn_test(4)
CPU times: user 483 µs, sys: 795 µs, total: 1.28 ms
Wall time: 4.42 s

For test.com connection will timeout after the ~same time as was specified in timeout argument. But for imap.mail.outlook.com things are getting interesting -- socket connection ignores timeout argument. To be precise - not ignores, but rather always timeouts connection after some greater period of time.

I may suppose that this behavior originates from imap.mail.outlook.com server, not from socket module.

Answer 1

First, you can unify your to functions into:

def fn_connect(host, port, timeout):
    try:
        s = socket.create_connection((host, port), timeout=timeout)
    except socket.timeout:
        return None
    else:
        return s

and call them like:

IMAP_HOST = "imap.mail.outlook.com"
IMAP_PORT = 993
TEST_HOST = "test.com"
TEST_PORT = 666
s1 = fn_connect(IMAP_HOST, IMAP_PORT, 2)
s2 = fn_connect(TEST_HOST, TEST_PORT, 1)
#and so on....

I returned the socket to be able to close it properly afterwards (if not None).

The problem lies in how the underlying socket mechanism resolves hostnames; create_connection calls getaddrinfo, and for each address returned it tries to create a socket and connect to it (and each socket has the very timeout you specified). So, the results for your 2 addresses:

• TEST_HOST, TEST_PORT

  socket.getaddrinfo("test.com", 666)
  [(2, 0, 0, '', ('69.172.200.235', 666))]
  

• IMAP_HOST, IMAP_PORT

  socket.getaddrinfo("imap.mail.outlook.com", 993)
  [(2, 0, 0, '', ('207.46.163.247', 993)),
   (2, 0, 0, '', ('207.46.163.138', 993)),
   (2, 0, 0, '', ('207.46.163.215', 993))]
  

As you can see, for IMAP_HOST, IMAP_PORT it returns 3 separate addresses (while for TEST_HOST, TEST_PORT it only returns one). Since you specified that none of it works it will attempt to connect to all of them resulting in a general timeout ~3 times bigger than you specified.