Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Recommendations for C/C++ remote message queues

I am working on a project which involves several C++ programs that each take input and generate output. The data (tens to hundreds of bytes, probably JSON) essentially flows (asynchronously) in one direction, and the programs will need to be located on different Linux computers around the LAN.

Since the data flows in only one direction, I don't believe I need a transactional model like HTTP. I think a message queue model (fire and forget) makes the most sense and should simplify the logic of each program. It is probably sufficient to merely note that the message was added to the remote queue successfully.

What I am looking for are recommendations for how to implement this message queue in C or C++. It seems like POSIX and Boost message queues are limited to a single host, and RabbitMQ seems to have weak C/C++ support, and MQ4CPP seems inadequately supported for a business-critical role. Am I wrong about this? What about Boost ASIO or ACE or writing socket code myself? I look forward to your suggestions.

like image 978
Randall Cook Avatar asked May 03 '12 22:05

Randall Cook


People also ask

How does Message Queuing improve the resilience of a distributed system?

Queues make your data persistent, and reduce the errors that happen when different parts of your system go offline. By separating different components with message queues, you create more fault tolerance. If one part of the system is ever unreachable, the other can still continue to interact with the queue.

Does message queues increase the complexity of the system architecture?

Implementing messages queues One thing to note is that decoupling of systems with message queues increases the complexity of the system's architecture, so that's one trade-off you will need to consider.

What is a message queue and when would one be useful?

Message queues allow different parts of a system to communicate and process operations asynchronously. A message queue provides a lightweight buffer which temporarily stores messages, and endpoints that allow software components to connect to the queue in order to send and receive messages.


1 Answers

In terms of simple messaging support, ZeroMQ is hard to beat. It's available in many language bindings and supports everything from simple send and receive to pub/sub, fanout, or even a messaging pipeline. The code is also easy to digest and makes it pretty easy to switch between patterns.

Looking at their Weather Update Server sample (in 20 some odd languages) shows how easy it can be to create publish/subscribe setups:

zmq::context_t context (1);
zmq::socket_t publisher (context, ZMQ_PUB);
publisher.bind("tcp://*:5556");
publisher.bind("ipc://weather.ipc");

while(1) {
    //  Send message to all subscribers
    zmq::message_t message(20);
    snprintf ((char *) message.data(), 20 ,
        "%05d %d %d", zipcode, temperature, relhumidity);
    publisher.send(message);
}

I've used it on some mixed C# and Python processes without much hassle.

like image 137
user7116 Avatar answered Sep 27 '22 19:09

user7116