Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python Requests SSL issue

It's been days now since I started to look for a solution for this.

I've been trying to use requests to make an https request through a proxy with no luck.

Altough this is included in a bigger project of mine, it all boils done to this:

    import requests

    prox = 'xxx.xxx.xxx.xxx:xxx' # fill in valid proxy

    r = requests.get('https://ipdb.at', proxies={'http': prox,
                                                 'https': prox})

I tried the kward verify=False but I keep getting the same error or a variation of it:

    Traceback (most recent call last):
      File "/Users/mg/Desktop/proxy_service.py", line 21, in <module>
        verify=False)
      File "/Library/Python/2.7/site-packages/requests/api.py", line 55, in get
        return request('get', url, **kwargs)
      File "/Library/Python/2.7/site-packages/requests/api.py", line 44, in request
        return session.request(method=method, url=url, **kwargs)
      File "/Library/Python/2.7/site-packages/requests/sessions.py", line 279, in request
        resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
      File "/Library/Python/2.7/site-packages/requests/sessions.py", line 374, in send
        r = adapter.send(request, **kwargs)
      File "/Library/Python/2.7/site-packages/requests/adapters.py", line 213, in send
        raise SSLError(e)
    requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
    [Finished in 20.4s with exit code 1]

I'm using the latest versions of requests and openssl and 2.7.3 for python. I tested it on mac mountain lion 10.8.2 and also on windows seven.

I googled this out, saw others having similar issues. I found related pull requests on the libraries requests and urllib3, also saw information about this being a bad implementation of the http connect verb. I found a fix to use a custom adapter (i don't recall exactly but I think it's the right term) to use a custom ssl protocol. Tried them all, none helped.

So, I'm looking for more info. Do you have any idea what is going on, I can I fix it, etc, etc...

All help is welcome and thank you all!

PS: I tried several proxies, and I'm sure they are not the issue.

like image 211
mg_ Avatar asked Jan 18 '13 00:01

mg_


People also ask

Does Python requests use SSL?

Requests verifies SSL certificates for HTTPS requests, just like a web browser. SSL Certificates are small data files that digitally bind a cryptographic key to an organization's details.

How do I stop SSL certificate warning message in Python?

This can be avoided by using urlib3. disable_warnings method. The above warning that occurred while using verify=False in the request method can be suppressed by using the urllib3. disable_warnings method.

How does Python handle SSL certificates?

Python by default just accepts and uses SSL certificates when using HTTPS, so even if a certificate is invalid, Python libraries such as urllib2 and Twisted will just happily use the certificate.


2 Answers

I had this same problem but when trying to use oauth2client (Google library for using oauth2 against their APIs). After a day of faffing ("experimentation"), I discovered that I had the env variable set as follows:

https_proxy=https://your-proxy:port

Changing this to

https_proxy=http://your-proxy:port

completely fixed the problem for me. Presumably this means that the initial handshake from the client to the proxy is now unencrypted, but after that the https must flow directly. I don't believe my company proxy was configured to talk https anyway.

Note however that my browsers all behaved fine with the previous setting. Perhaps they silently tried to establish the proxy connection over http after failing to use https.

Googling shows examples of setting https_proxy to both http or https addresses in equal measure. I guess it depends on how the proxy is configured.

The "use for all protocols" button in browser/OS configuration pop-ups makes it easy to accidentally use the setting I had.

However I'm not sure if that is your problem. Your code suggests that you already use exactly the same proxy setting for both http and https.

By the way, this was all on Ubuntu 12.04.1.

like image 128
Timothy Corbett-Clark Avatar answered Sep 18 '22 15:09

Timothy Corbett-Clark


The answer here worked for me using requests with TLS doesn't give SNI support

I Installed the three dependent libraries and included the monkey patch tip in the second answer

like image 43
brianf Avatar answered Sep 19 '22 15:09

brianf