twisted http client

Question

I am after an example describing the usage of Twisted's HTTP Client.

After reading the excellent blog post on the internals of Twisted, I understand how the "Factory" and "Protocol" components play their role but I am unclear on how to introduce "Request" in the overall Client flow.

More specifically, I need to be able to perform HTTP GET and POST requests to a remote server using Twisted.

Updated: after a discussion on irc #twisted / #python, it seems that twisted.web2 is fading away in favor of beefing up functionality on twisted.web e.g. Agent.

Answer 1

As of Twisted 9.0, there are actually two HTTP clients available. The older one has quite a few features, such as automatically following redirects, interpreting cookie headers, etc. You can find an example of its usage here:

http://twistedmatrix.com/documents/current/web/examples/

(getpage.py and dlpage.py)

Unfortunately, the interface presented by the older client makes a number of common tasks difficult. For example, using getPage, you cannot examine arbitrary response headers.

The newer HTTP client isn't yet as featureful as the old one, but it presents an interface intended to eliminate the limitations of getPage. It is also intended to be more easily extended and customized. You can find a document describing its usage here:

http://twistedmatrix.com/documents/current/web/howto/client.html

Answer 2

I started using treq with twisted. treq has an API which is very similar to Requests. https://pypi.python.org/pypi/treq/0.2.0