Socket-timeout: timed out python requests

Question

Using the requests python lib, I make a GET request, and handle Timeout exceptions (as well as other exceptions I don't show here) like

import requests
timeout1=20
timeout2=40

try:
    #first attempt
    resp = requests.get(base_url+resource, params=payload, headers=headers,
    timeout=timeout1)
except requests.exceptions.Timeout:
    #timed out, retry once
    try:
       resp = requests.get(base_url+resource, params=payload, headers=headers,
       timeout=timeout2)
       return resp.json()
    except requests.exceptions.RequestException as e:
       #Still failed; return error code
       return -1

This works fine most of the time, however sometimes my program just completely exits with the error socket.timeout: timed out, instead of throwing the requests.exceptions.Timeout and this being caught and dealt with.

Why is the requests lib behaving like this? How should I handle this?

Answer 1

I think this is an issue of the urllib3 that requests uses:

https://github.com/kennethreitz/requests/issues/1236

You'll have to catch that socket.timeout error. (based on jb's comment at Correct way to try/except using Python requests module?)