ZeroMQ distribution pattern

Question

I currently have a pub/sub system running which allows clients to connect to a central message routing daemon, subscribe for a range of messages, and then start chattering away. The routing daemon tracks and maintains each subscriber's messages of interest (based on a simple tag) and delivers the appropriate messages of interest as each of the subscribers produce them. Essentially, each connection is considered a potential publisher OR subscriber AND USUALLY both, the daemon handles the routing and delivery as needed.

For example, three clients all connect and subscribe for their message tag(s) (MT) of interest:

Client 1(C1) subscribes to MT => 123
Client 2(C2) subscribes to MT => 123 & 456
Client 3(C3) subscribes to MT => 123 & 456 & 789

C1 produces MT 456: daemon delivers a copy to C2 and C3
C2 produces MT 123: daemon delivers a copy to C1 and C3 (not self)
C3 produces MT 999: daemon delivers it to none (nobody subscribed)

ZeroMQ came up in a discussion with a coworker and after tinkering with it for a few days I don't think I'm seeing the proper pattern for implementing/replacing the system that we currently have in place. Additionally, I would like to use EPGM in order to take advantage of the multicast gains and to eliminate the TCP based daemon, monkey in the middle, that I currently have.

Any suggestions?

Answer 1

It's possible to design a system like that using ZeroMQ. Basically speaking, you may create a daemon that binds two sockets: PULL to receive messages from clients and PUB to publish messages. Each of clients connects SUB socket and PUSH socket to server. EPGM might be used for PUB/SUB sockets, but PUSH/PULL sockets are still TCP.

The disadvantage of this design is that topic filtering and dropping out own messages must be done manually. For example, you might create message of three parts:

1. Topic
2. ID of producer
3. Message body

Client should read messages part by part immediately dropping tail of message it's not interested in. Working with PUB/SUB message envelopes is described in detail in this section of the guide: http://zguide.zeromq.org/page:all#Pub-Sub-Message-Envelopes. Client filtering shouldn't affect performance, since all PGM packets must be delivered to all connected receivers anyway.

This design is very simple yet pretty effective. It doesn't cover reliability, high availability, failure recovery and other important aspects - it's all doable with ZeroMQ and covered in the guide. Probably the best feature of ZeroMQ is the ability to start with something simple and add functionality as necessary without pain and/or major rewrites.

Something very similar (plus state snapshots, reliability and many more) is described in the chapter "Reliable Pub-Sub (Clone Pattern)" of the guide: http://zguide.zeromq.org/page:all#toc119

BTW, it's also possible to design p2p system with the central daemon used only as a name server, but it will be definitely more complex.