Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

zmq: can multiple threads PUSH in a simple PUSH-PULL pattern

I have two processes: a producer which pushes messages via ZMQ to a consumer in a simple PULL-PUSH point-to-point pattern. The producer has several internal threads that send() via zmq. However, 0MQ's docs suggest not to share sockets between threads.

Must I use a single thread to send?
Assuming there is no strict requirement for keeping the sending order between the threads, doesn't the fact that the socket is a one-directional simplex allow multiple threads to use it without introducing locks?

like image 502
Jonathan Livni Avatar asked Jul 14 '13 12:07

Jonathan Livni


People also ask

What is ZMQ socket?

ZeroMQ (also known as ØMQ, 0MQ, or zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast.

What is ZMQ context?

ZMQ. Context Context is an object serving as a container for all the sockets of a single process. By creating a new context, you start one or more input/output threads: DEFINE context ZMQ.Context. Associated methods: socket()

What is the difference between publisher subscriber and push pull model?

The difference is that a PUB socket sends the same message to all subscribers, whereas PUSH does a round-robin amongst all its connected PULL sockets. In your example, if you send just a single message from the root, then all the subscribers will receive it (barring slow subscribers, etc.)

What is ZMQ Python?

ZeroMQ (also spelled ØMQ, 0MQ or ZMQ) is a high-performance asynchronous messaging library, aimed at use in distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can run without a dedicated message broker.


1 Answers

The easiest thing to do is to create a separate PUSH socket on each of producer's threads and connect all these sockets to a single PULL socket in consumer.

It's explicitly stated in the guide that ZeroMQ sockets must be used on a single thread. I'd say that violating this requirement is not a good idea, even if it seems to work: things may break in the next version of the library or on some specific platform or in some specific load scenario. So, it's just too risky.

like image 162
Wildfire Avatar answered Oct 18 '22 17:10

Wildfire