What's the best way to send an object over a network in Python?

Question

I need to send objects around a network. I'm going to be using Twisted, and I've just started looking around the documentation for it.

As far as I know, the only way python implements sockets is through text. So how would I send an object using strings? Pickle? Or is there something better?

Answer 1

The most general serialization on offer between Python end-points is the pickle format (in Python 2.any, be sure to use the cPickle module, and the -1 aka pickle.HIGHEST_PROTOCOL protocol; if you need interoperability between Python 2.any and Python 3.any more care is needed). For especially simple objects, the marshal module can sometimes be faster and more compact. For interoperation with non-Python endpoints, json may be best (or you could use xml to define or adopt other existing serialization formats), but those will likely be bulkier and slower to format and parse.

As far as I know, the only way python implements sockets is through text.

Nope, all strings of bytes are welcome!-) You may be confused by the fact that in Python 2 a "normal string" is actually a string of bytes ("text" would be the unicode type); Python 3 sets things right and uses Unicode for "normal strings" and a specific byte string type for strings of bytes.

Strings of bytes are the general way in which any language will perform any form of serialization and deserialization, according to some protocol or other -- such byte streams or blobs can go into networks, databases, plain files, etc, etc, of course.

Twisted offers its own serialization format, as part twisted.spread -- it's mostly for use with Perspective Broker (PB) but you could repurpose it for your own purposes if you don't want to use PB for some special reason. The docs for the serialization part, twisted.spread.jelly, are here, and the summarize well the format's goals...:

S-expression-based persistence of python objects.

It does something very much like Pickle; however, pickle's main goal seems to be efficiency (both in space and time); jelly's main goals are security, human readability, and portability to other environments.

If you care more about security, readability, and portability, than speed and compactness, then jelly might indeed serve you well.