How do CPG of Corosync, ZeroMQ, and Spread compare for messaging?

Question

I'm interested in:

• Performance
  • Latency
  • Throughput
  • Resource usage (CPU, memory, ...)
• High availability
  • No single point of failure
• Features
  • Transport options
  • Routing options
• Stability
• Community
  • Active development
  • Widely used
  • Helpful mailing list, forum, IRC channel, ...
• Ease of integration with my current codebase
• Gotchas maybe
• Any other thing you think I omitted

I've read about them, but I couldn't find a good comparison. Specially I'm interested in performance benchmarks comparing them. (Maybe I should do one on my own! I hope not.)

Answer 1

Well, I haven't used the other two, but can share my experiences with ZeroMQ. In my opinion, it excels at all of yours.

Speed and throughput

It's as fast as TCP, doesn't use CPU or a lot a memory. It can push A LOT of messages very quickly without a sweat. It will saturate your network channel way before you run out of memory (I doubt you'll ever be able to max-out the CPU). There was a comparison to RabbitMQ somewhere and ZMQ outperforms it by a factor of 2. From things I've read around the web it's in use in high speed trading.

RabbitMQ is also a very good tool. Have a look at it - it might be good fit for what you are looking

SPOF

If you design you application properly, then you can have no single point of failure. It's very easy to connect two sockets to another one. So if one of them fails - the other is there to handle the work. There are things like High water marks to help you along the way. Read the ZeroMQ Guide to learn how to design your app without a SPOF.

Transports and routing

Regarding transport options (if I'm understanding this correctly) - it's up to you to define your protocol. ZeroMQ basically promises you that it will deliver this blob of data to the other end. Use JSON, Protocol buffers, Morse code, whatever you like.

There is no built-in routing in like there is in AMQP. Again, it up to you to specify which ZeroMQ socket connects to which, but this is very easy.

Stability

I've been developing with it for a few months (using Python) and haven't found a single issue with its stability. Even when I try to use it the wrong way it just throws a nice error telling me not to do that. Even restarting/killing some of the services and bringing them back up doesn't cause any problems. I'd say it a very stable piece of software.

As a note: always use the latest version - the 2.1 version is very much stability oriented, so many stability issues are resolved in it.

Community

Bindings for more than 20 languages, active mailing list, very good documentation, frequent releases. Anything else?

Integration

Because it's designed as a library it's up to you to design you application (unlike the case with a framework) and it pretty much stands out of your way. It feels a bit like a normal TCP socket, much more powerful and easier to use (it guarantees you that a message will be delivered as a whole, not only the first 128 bytes and the rest later as it the case with regular sockets).

Gotchas

There are some, but they are all documented in the guide. (For example: you might miss the first few messages from a PUB socket when you connect (SUB) to it. There is an explanation to this in the guide and a recipe how to handle it).

Overall

I find this one of the best designed pieces of software - stable, well written, well documented and doesn't stand in my way.

I recommend you to read the guide end-to-end. It's well written, examples in a lot of languages (including C++) and it describes a lot of edge cases and pain points.