Socks proxy in tornado AsyncHttpClient

Question

How can I use socks proxy in tornado AsyncHttpClient? I found it possible to use only HTTP Proxy without changing the lib...

Answer 1

According to the documentation, proxy support is only available for the libcurl implementation of AsyncHTTPClient.

If you will take a deeper look at the HTTPRequest object you're passing to the fetch() method, you'll notice there's an extra prepare_curl_callback argument, which can call setopt on the PyCurl object before the request is sent.

Here's a little example of such prepare_curl_callback function:

import pycurl

def prepare_curl_socks5(curl):
    curl.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5)

And a full usage example:

import tornado
import tornado.ioloop
import tornado.gen
import tornado.httpclient
import pycurl

def prepare_curl_socks5(curl):
    curl.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5)

@tornado.gen.coroutine
def main():
    # set CurlAsyncHTTPClient the default AsyncHTTPClient
    tornado.httpclient.AsyncHTTPClient.configure(
        "tornado.curl_httpclient.CurlAsyncHTTPClient")

    http_client = tornado.httpclient.AsyncHTTPClient()
    http_request = tornado.httpclient.HTTPRequest(
        "http://jsonip.com",
        prepare_curl_callback=prepare_curl_socks5,
        proxy_host="localhost",
        proxy_port=9050
    )
    response = yield http_client.fetch(http_request)

    print response.body

if __name__ == '__main__':
    tornado.ioloop.IOLoop.instance().run_sync(main)

The additional keyword argument prepare_curl_callback=prepare_curl_socks5 to the fetch() call does the magic, making CURL use SOCKS5 proxy instead of the default - HTTP proxy.