Message Bus vs. Multicast

Question

I am working on an application which consists of several modules and requires them to share information with each other. Example: publish/subscribe scenario where a module publishes some information (e.g. a state variable) and modules interested in the particular information get it. Or a request/reply scenario where the interested module explicitly asks about the information and got answered.

I have been looking into different message bus implementations, namely D-bus, ØMQ, RabbitMQ and QPID (the latter two are based on AMQP). However then someone pointed out that instead of trying some complex and heavy message bus implementation, why don't I simply use multicast to solve the problem.

Lacking the experience to see if multicast can really solve my problem and to understand the pros and cons of the two solutions, I would kindly request experts to help me out. Would very much appreciate.

Answer 1

Having experience with both message buses and multicast in a large, high-volume production environment, and having talked with several experienced network engineers about the problems, I can say that you should avoid multicast like the plague unless you are broadcasting to very high numbers of nodes (hundreds).

If you're going to use multicast you have to understand that it is an unreliable protocol. Messages can go missing, they can be duplicated, and so on. You'll need to spend a lot of time getting the reliability protocol (retries, duplicate detection, resends) on top of multicast correct for it to be useful. There is a good anecdote about an army test of multicast commanded robotic tanks that I'm trying to find a reference for... basically when you are sending "turn right 90 degrees, turn right 90 degrees, fire" to a line of tanks and some of them receive only 1 turn right message and others get 3, it's a recipe for mayhem.

Depending on the sort of information you need to share there are several options.

If they are sharing configuration information, look at something like Zookeeper. It's reliable, lightweight, and simple to use. The most recent values of the shared state are always available and persisted. With a message bus you still need to have a resend protocol in case your module misses the last configuration message by being down.

With respect to a message bus, they can be complex. However, I would not put ZeroMQ in that category necessarily. It can emulate a message bus, but it's more of a point to point mechanism. I have not used it in production but the research and prototyping I've done with it was very favorable.

Another option might be a distributed data grid like Oracle Coherence, GridGain, GigaSpaces, and so on. Again, this is another application to install and maintain so your complexity goes up, but there are many uses for a data grid.

Another MQ option is HornetMQ. I haven't used it (we use two commercial MQs in-house, both Sonic and MQ Series), but I've seen some favorable comparisons.

D-Bus seems to be optimized for communication on a single machine, and the FAQ recommends looking elsewhere if you are doing peer-to-peer, clustering, or other similar things. Caveat: I have never used D-Bus so I'm basically regurgitating information I just read.

Answer 2

Are you worried about packets/messages being dropped or lost? Message buses can handle or mitigate those concerns while multicast won't by default.