Python web service with Twisted

Question

This is connected with my previous question Python web service.

I'll use Tornado to exchange information between server and clients. There will be one server and N clients. Clients will send information (disk usage, processes etc.) periodically (every 2 minutes or so). The data on client side will be represented by custom classes/lists. It would be nice to have the same data on the other side (server).

I have experience with SOAP and it would probably be ok for this (with enough bit timeouts on the server), but would rather use something more light and pythonic. The communication will be more or less only client -> server.

Both server and client side are writen in Python.

What should I look for in Twisted documentation for doing this kind of stuff?

edit: I'm not asking on how to serialize data (JSON or pickle or XML etc.). I would like to know what are the options of Twisted to transport data.

With SOAP I would have methods like this:

- sendDiskUsage(DiskUsage class instance)
- sendProcesses(ProcessList class instance)
- etc..

I would like to know what are the options with Twisted. One of them is XML-RPC which would be ok, but it's not my favourite...

edit2: the communication will be "two way" - client's will read tasks from server...

Answer 1

I recommend AMP. It's a very simple key-value pair based protocol, ideal for what you're doing. Perspective broker is another alternative.. but it's slightly complicated, and usually unnecessary.

AMP runs directly over TCP (why bother with HTTP?), the serialization format is both minimal and logical. This makes it 'light' and 'pythonic' in the sense you probably mean, but those terms could be interpreted in a few different ways.

Take a look at the AMP examples on Twisted code examples, they're pretty self-explanatory. AMP connections are bi-directional, so try modifying the example so that the server asks the client for its current disk usage. The twisted.protocols.amp API docs will be useful here.

When you're ready to build a real application read Twisted from Scratch, or The Evolution of Finger.

AMP has been implemented in most of the popular languages, but if you're looking for something more 'mainstream', protobuf is Google's thing. IMO it's overcomplicated and lacks some important features like error feedback.

Answer 2

You could try twisted's “perspective broker“. It has a few nice features, such as symmetry (there is no real difference betweenclient and server, once a connection is established). It takes care of serialization itself. It might not be the best choice, if you only want a simple one-way status update push.

Answer 3

For data serializing, if you are using pickle to serialize custom class instances, do make sure that both the server and the client have the same class definition in the global namespace.

For twisted reference, you may take a look at its documentation, especially the 'Writing a TCP server' and 'Writing a TCP client' part.