ZeroMQ: asynchronous replies

Question

I'm building an infrastructure (in C) where several seperate services communicate trough ZeroMQ. Now, in order to make this all running fast and smoothly, I want ZeroMQ to run asynchronously.
I know that ZeroMQ's IO-thread already is separated, but I want to perform multiple tasks at the same time.

At the moment, my pattern looks like this:

Client -> REQ     |  /\
Router -> ROUTER  |   |
---   proxy   --- |   |
Dealer -> DEALER  |   |
Workers-> REP     \/--|

However, this requires me to design a blocking, synchronous worker. The only way to make this asynchronous is to increase the number of workers (threads), but this doesn't seem very scalable to me.

Furthermore, the tasks that the workers perform are asynchronous themselves (using their own event loop). So basically, I'm looking to achieve something like this:

void *socket = zmq_socket(context, ZMQ_REP);
zsocket_connect(socket, "inproc://backend");

while (1) {
    zmq_msg_t request;
    zmq_msg_init(&request);
    zmq_msg_recv(&request, socket, 0);
    zmq_msg_close(&request);

    do_something(callback, socket);
}

do_something would then perform some operations, and when it's done, the callback function gets called.

void callback(void *data, void *socket) {
    zmq_msg_t reply;
    zmq_msg_init_size(&reply, 5);
    memcpy(zmq_msg_data(&reply), "World", 5);
    zmq_msg_send(&reply, socket, 0);
    zmq_msg_close(&reply);
}

Now, is there any way to achieve this (i.e. to combine two event loops)?

I've already looked into DEALER (client) -> ROUTER -> DEALER -> DEALER (worker), but that doesn't seem to work either, because it still blocks concurrent tasks at the worker end.

Answer 1

(client) DEALER<->ROUTER |<->| ROUTER <-> DEALER (worker) In this connection pattern nothing will block, hence you can interleave receives and sends however you want.