requests via a SOCKs proxy

Question

How can I make an HTTP request via a SOCKs proxy (simply using ssh -D as the proxy)? I've tried using requests with SOCK proxies but it doesn't appear to work (I saw this pull request). For example:

proxies = { "http": "socks5://localhost:9999/" }
r = requests.post( endpoint, data=request, proxies=proxies )

It'd be convenient to keep using the requests library, but I can also switch to urllib2 if that is known to work.

Answer 1

Since SOCKS support has been added to requests 2.10.0, it is remarkably simple, and very close to what you have

1. Install requests[socks]:

   $ pip install requests[socks]
   

2. Set up your proxies variable, and make use of it:

   >>> import requests
   >>> proxies = {
           "http":"socks5://localhost:9999",
           "https":"socks5://localhost:9999"
       }
   >>> requests.get(
           "https://api.ipify.org?format=json",
           proxies=proxies
       ).json()
   {u'ip': u'123.xxx.xxx.xxx'}
   

A few things to note are to not use a / on the end of the proxies URL, and that you can also use socks4:// as the scheme too if the SOCKS server doesn't support SOCKS5.

Answer 2

SOCKS support for requests is still pending. If you want, you can view my Github repository here to see my branch of the Socksipy library. This is the branch that is currently being integrated into requests; it will be some time before requests fully supports it, though.

https://github.com/Anorov/PySocks/

It should work okay with urllib2. Import sockshandler in your file, and follow the example inside of it. You'll want to create an opener like this:

opener = urllib2.build_opener(SocksiPyHandler(socks.PROXY_TYPE_SOCKS5, "localhost", 9050))

Then you can use opener.open(url) and it should tunnel through the proxy.