Python - requests.exceptions.SSLError - dh key too small

Question

I'm scraping some internal pages using Python and requests. I've turned off SSL verifications and warnings.

requests.packages.urllib3.disable_warnings()
page = requests.get(url, verify=False)

On certain servers I receive an SSL error I can't get past.

Traceback (most recent call last):
  File "scraper.py", line 6, in <module>
    page = requests.get(url, verify=False)
  File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/api.py", line 71, in get
    return request('get', url, params=params, **kwargs)
  File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/api.py", line 57, in request
    return session.request(method=method, url=url, **kwargs)
  File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/cygdrive/c/Users/jfeocco/VirtualEnv/scraping/lib/python3.4/site-packages/requests/adapters.py", line 477, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: SSL_NEGATIVE_LENGTH] dh key too small (_ssl.c:600)

This happens both in/out of Cygwin, in Windows and OSX. My research hinted at outdated OpenSSL on the server. I'm looking for a fix client side ideally.

Edit: I was able to resolve this by using a cipher set

import requests

requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += 'HIGH:!DH:!aNULL'
try:
    requests.packages.urllib3.contrib.pyopenssl.DEFAULT_SSL_CIPHER_LIST += 'HIGH:!DH:!aNULL'
except AttributeError:
    # no pyopenssl support used / needed / available
    pass

page = requests.get(url, verify=False)

Answer 1

this is not an extra answer just try to combine the solution code from question with extra information So others can copy it directly without extra try

It is not only a DH Key issues in server side, but also lots of different libraries are mismatched in python modules.

Code segment below is used to ignore those securitry issues because it may be not able be solved in server side. For example if it is internal legacy server, no one wants to update it.

Besides the hacked string for 'HIGH:!DH:!aNULL', urllib3 module can be imported to disable the warning if it has

import requests
import urllib3

requests.packages.urllib3.disable_warnings()
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
try:
    requests.packages.urllib3.contrib.pyopenssl.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
except AttributeError:
    # no pyopenssl support used / needed / available
    pass

page = requests.get(url, verify=False)

Answer 2

Disabling warnings or certificate validation will not help. The underlying problem is a weak DH key used by the server which can be misused in the Logjam Attack.

To work around this you need to chose a cipher which does not make any use of Diffie Hellman Key Exchange and thus is not affected by the weak DH key. And this cipher must be supported by the server. It is unknown what the server supports but you might try with the cipher AES128-SHA or a cipher set of HIGH:!DH:!aNULL

Using requests with your own cipher set is tricky. See Why does Python requests ignore the verify parameter? for an example.

Answer 3

This also worked for me:

import requests
import urllib3
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'ALL:@SECLEVEL=1'

openssl SECLEVELs documentation: https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_security_level.html

SECLEVEL=2 is the openssl default nowadays, (at least on my setup: ubuntu 20.04, openssl 1.1.1f); SECLEVEL=1 lowers the bar.

Security levels are intended to avoid the complexity of tinkering with individual ciphers.

I believe most of us mere mortals don't have in depth knowledge of the security strength/weakness of individual ciphers, I surely don't. Security levels seem a nice method to keep some control over how far you are opening the security door.

Note: I got a different SSL error, WRONG_SIGNATURE_TYPE instead of SSL_NEGATIVE_LENGTH, but the underlying issue is the same.

Error:

Traceback (most recent call last):
  [...]
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='somehost.com', port=443): Max retries exceeded with url: myurl (Caused by SSLError(SSLError(1, '[SSL: WRONG_SIGNATURE_TYPE] wrong signature type (_ssl.c:1108)')))