check Python requests with charles proxy for HTTPS

Question

I want to debug some python requests using charles proxy.

I need to include the certificate for charles on the call, but is not working

import requests
endpoint_url = 'https://www.httpsnow.org/'
r = requests.get(endpoint_url, verify=True, cert='/Users/iosdev/DopPy/charles.crt')

print "empexo"
print r

I have added the https address on Charles,

I get on Charles:

SSLHandshake: Remote host closed connection during handshake

and on python the log with error

empexo
Traceback (most recent call last):
  File "/Users/iosdev/DopPy/GetCelebs.py", line 15, in <module>
    r = requests.get(endpoint_url, verify=True, cert='/Users/iosdev/DopPy/charles.crt')
  File "/Users/iosdev/VenvPY26/lib/python2.6/site-packages/requests/api.py", line 65, in get
    return request('get', url, **kwargs)
  File "/Users/iosdev/VenvPY26/lib/python2.6/site-packages/requests/api.py", line 49, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/Users/iosdev/VenvPY26/lib/python2.6/site-packages/requests/sessions.py", line 461, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/iosdev/VenvPY26/lib/python2.6/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/Users/iosdev/VenvPY26/lib/python2.6/site-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 336265225] _ssl.c:341: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib

Process finished with exit code 1

Answer 1

I found this thread while I was troubleshooting a similar issue. In the scenario I ran into the cert argument was being used to define the path to a ".crt" file when the verify argument should have been used instead.

The correct usage ended up looking like:

requests.get(endpoint_url, verify='/path/to/file.crt')

See Requests' documentation for more details: https://2.python-requests.org/en/v1.1.0/user/advanced/#ssl-cert-verification

---

As an aside, I find employing Request's ability to specify the path to a ".crt" via the REQUESTS_CA_BUNDLE environmental variable more effective when using Charles Proxy for local debugging.

Running something like the following in shell saves having to specify the path to Charles' ".crt" for every Requests call:

REQUESTS_CA_BUNDLE=/path/to/file.crt
export REQUESTS_CA_BUNDLE