How can I access a peer's cert chain from a python-requests response/exception object?

Question

I use python-requests to talk to HTTPS web services, some of which present incomplete certificate X509 chains. I'm having trouble figuring out how to access the invalid/incomplete certificates in order to explain the error to the user.

Here's an example illustrated by https://ssllabs.com/ssltest, where the server sends only the leaf certificate, and not the intermediate certificate which is necessary for validation, but missing from certifi's root CA store:

When I try to connect with python-requests, I get an exception that isn't very useful:

request.get('https://web.service.com/path')

SSLError: HTTPSConnectionPool(host='web.service.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

Obviously, I can use separate tools to figure out what's wrong in any particular case (e.g. gnutls-cli, openssl s_client, SSLLabs, etc.).

However, what I really want to be able to do is to be able to catch and diagnose the problem with the certificate chain in my Python code, so that I can present a more specific error message to the user. This answer suggests a monkey-patch to the response object; it's not particularly elegant, but it works—though only when the response object is returned successfully, and not in the case of an exception.

What are the cleanest ways to instrument requests to save the peer's certificate chain in the exception object returned when requests fails to validate the certificate chain itself?

Answer 1

Take requests.get("https://151.101.1.69") # stackoverflow's ip as an example:

try:
    requests.get("https://151.101.1.69")
except requests.exceptions.SSLError as e:
    cert = e.args[0].reason.args[0]._peer_cert

Then cert is a dict contains the peer's certificate. As I'm not very familiar with SSL, I don't know if it is enough for your case.

BTW, in this case the error is "hostname '151.101.1.69' doesn't match either of '*.stackexchange.com', ...omitted. I'm not sure about the structure of exception in your real case, so you may need to find it on your own. I think it should have the same name _peer_cert.

update

The above method doesn't work when handshake fails... But it still can be done:

try:
    requests.get("https://fpslinux1.finalphasesystems.com/")
except requests.exceptions.SSLError:
    import ssl
    import OpenSSL
    cert = ssl.get_server_certificate(('fpslinux1.finalphasesystems.com', 443))
    cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
    print(cert.get_issuer())
    print(cert.get_subject().get_components())

Yes it is a little dirty but I don't have a better method as a ssl socket doesn't even return invalid certs from C level :/

To use OpenSSL, you need to install pyopenssl.