"requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))"

Question

I'm trying to connect to https://apis.digital.gob.cl/fl/feriados/2020, but I get an requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',)) error on a script that works perfectly with other URLs.

The code:

import requests

response = requests.get('https://apis.digital.gob.cl/fl/feriados/2020')
print(response.status_code)

Answer 1

The issue is that the website filters out requests without a proper User-Agent, so just use a random one from MDN:

requests.get("https://apis.digital.gob.cl/fl/feriados/2020", headers={
"User-Agent" : "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
})

Answer 2

It might be due to idle timeout. Overriding default socket options can help

import socket
from urllib3.connection import HTTPConnection

HTTPConnection.default_socket_options = (
    HTTPConnection.default_socket_options + [
        (socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),
        (socket.SOL_TCP, socket.TCP_KEEPIDLE, 45),
        (socket.SOL_TCP, socket.TCP_KEEPINTVL, 10),
        (socket.SOL_TCP, socket.TCP_KEEPCNT, 6)
    ]
)