Python <-> C++ object oriented communication

Question

I want to write a daemon in C++, which will hold a graph data structure and will compute some dependencies. I also want to have Python Batch (also a daemon - a backend to HTML based GUI), which will allow users to interactivly operate on these C++ structures - add / remove / connect / ... nodes and read results of computations.

I would love to choose the best communication mechanism available.

The mandatory functionality is:

1. Python and C++ should be able to operate on nodes in object oriented way, so I would love to be able to write code like n1 = node('a'); n2 = n1.add_subnode('b'); n2.ports('test').connect(node('c'))
2. The Python Batch does NOT have to be "separated" from the C++ daemon - they can have the same lifetime (but it would be good to somehow separate batch from C++ daemon in case of C++ crash or something wrong - this separation is optional)
3. The communication should be fast - the Python should be able to get information about a lot of nodes and allow end - users to work smoothly as much as possible.

Currently I was thinking about:

1. IPC (like 0MQ) with some kind of data serialization mechanism.
2. RPC based on Protocol Buffers or Thrift.
3. Integration based on Boost.Python

The IPC and RPC solutions seems good, but I have to write a big wrappers to get the functionality from point 1. On the other hand I have found no information about using Boost.Python in C++ daemon and I don't know if it is even possible.

Answer 1

1. Boost.Python may be used in a daemon.

2. Thrift and Protocol Buffers work fine. Thrift implements a full RPC server while protobuf, unless the situation has changed last year, just offers serialization. Personally I would prefer Thrift.

The differences between these two solutions are speed (Boost.Python is definitely faster, though RPC is not really slow if you specify correct socket options -- TCP_NODELAY etc.) and the fact that in case of Boost.Python your binary depends on a certain version of Python. In case of Thrift you have less dependencies, especially if the Thrift itself is installed as a package for your OS distribution. Anyway, this is a question of performance and deployment. It cannot be answered without knowing how fast the communication should be, and also where and how you are going to deploy your program.

UPD: Do you really need to write your daemon in C++? If that is because of heavy computation performed on graph, maybe only the computational part should be in C++ (an extension module)? Extending is usually preferred to other techniques.