How to get response SSL certificate from requests in python?

Question

Trying to get the SSL certificate from a response in requests.

What is a good way to do this?

Answer 1

requests deliberately wraps up low-level stuff like this. Normally, the only thing you want to do is to verify that the certs are valid. To do that, just pass verify=True. If you want to use a non-standard cacert bundle, you can pass that too. For example:

resp = requests.get('https://example.com', verify=True, cert=['/path/to/my/ca.crt']) 

Also, requests is primarily a set of wrappers around other libraries, mostly urllib3 and the stdlib's http.client (or, for 2.x, httplib) and ssl.

Sometimes, the answer is just to get at the lower-level objects (e.g., resp.raw is the urllib3.response.HTTPResponse), but in many cases that's impossible.

And this is one of those cases. The only objects that ever see the certs are an http.client.HTTPSConnection (or a urllib3.connectionpool.VerifiedHTTPSConnection, but that's just a subclass of the former) and an ssl.SSLSocket, and neither of those exist anymore by the time the request returns. (As the name connectionpool implies, the HTTPSConnection object is stored in a pool, and may be reused as soon as it's done; the SSLSocket is a member of the HTTPSConnection.)

So, you need to patch things so you can copy the data up the chain. It may be as simple as this:

HTTPResponse = requests.packages.urllib3.response.HTTPResponse orig_HTTPResponse__init__ = HTTPResponse.__init__ def new_HTTPResponse__init__(self, *args, **kwargs):     orig_HTTPResponse__init__(self, *args, **kwargs)     try:         self.peercert = self._connection.sock.getpeercert()     except AttributeError:         pass HTTPResponse.__init__ = new_HTTPResponse__init__  HTTPAdapter = requests.adapters.HTTPAdapter orig_HTTPAdapter_build_response = HTTPAdapter.build_response def new_HTTPAdapter_build_response(self, request, resp):     response = orig_HTTPAdapter_build_response(self, request, resp)     try:         response.peercert = resp.peercert     except AttributeError:         pass     return response HTTPAdapter.build_response = new_HTTPAdapter_build_response 

That's untested, so no guarantees; you may need to patch more than that.

Also, subclassing and overriding would probably be cleaner than monkeypatching (especially since HTTPAdapter was designed to be subclassed).

Or, even better, forking urllib3 and requests, modifying your fork, and (if you think this is legitimately useful) submitting pull requests upstream.

Anyway, now, from your code, you can do this:

resp.peercert 

This will give you a dict with 'subject' and 'subjectAltName' keys, as returned by pyopenssl.WrappedSocket.getpeercert. If you instead want more information about the cert, try Christophe Vandeplas's variant of this answer that lets you get an OpenSSL.crypto.X509 object. If you want to get the entire peer certificate chain, see GoldenStake's answer.

Of course you may also want to pass along all the information necessary to verify the cert, but that's even easier, because it already passes through the top level.

Answer 2

To start, abarnert's answer is very complete. While chasing the proposed connection-close issue of Kalkran I actually discovered that the peercert didn't contain detailed information about the SSL Certificate.

I dug deeper in the connection and socket info and extracted the self.sock.connection.get_peer_certificate() function which contains great functions like:

• get_subject() for CN
• get_notAfter() and get_notBefore() for expiration dates
• get_serial_number() and get_signature_algorithm() for crypto related technical details
• ...

Note that these are only available if you have pyopenssl installed on your system. Under the hood, urllib3 uses pyopenssl if it's available and the standard library's ssl module otherwise. The self.sock.connection attribute shown below only exists if self.sock is a urllib3.contrib.pyopenssl.WrappedSocket, not if it's a ssl.SSLSocket. You can install pyopenssl with pip install pyopenssl.

Once that's done, the code becomes:

import requests  HTTPResponse = requests.packages.urllib3.response.HTTPResponse orig_HTTPResponse__init__ = HTTPResponse.__init__ def new_HTTPResponse__init__(self, *args, **kwargs):     orig_HTTPResponse__init__(self, *args, **kwargs)     try:         self.peer_certificate = self._connection.peer_certificate     except AttributeError:         pass HTTPResponse.__init__ = new_HTTPResponse__init__  HTTPAdapter = requests.adapters.HTTPAdapter orig_HTTPAdapter_build_response = HTTPAdapter.build_response def new_HTTPAdapter_build_response(self, request, resp):     response = orig_HTTPAdapter_build_response(self, request, resp)     try:         response.peer_certificate = resp.peer_certificate     except AttributeError:         pass     return response HTTPAdapter.build_response = new_HTTPAdapter_build_response  HTTPSConnection = requests.packages.urllib3.connection.HTTPSConnection orig_HTTPSConnection_connect = HTTPSConnection.connect def new_HTTPSConnection_connect(self):     orig_HTTPSConnection_connect(self)     try:         self.peer_certificate = self.sock.connection.get_peer_certificate()     except AttributeError:         pass HTTPSConnection.connect = new_HTTPSConnection_connect 

You will be able to access the result easily:

r = requests.get('https://yourdomain.tld', timeout=0.1) print('Expires on: {}'.format(r.peer_certificate.get_notAfter())) print(dir(r.peer_certificate)) 

If, like me, you want to ignore SSL Certificate warnings just add the following in the top of the file and do not SSL verify:

from requests.packages.urllib3.exceptions import InsecureRequestWarning requests.packages.urllib3.disable_warnings(InsecureRequestWarning)  r = requests.get('https://yourdomain.tld', timeout=0.1, verify=False) print(dir(r.peer_certificate))