what is the expected performance of ZeroMQ?

Question

I am dabbling with process-to-process communication; the aim is to have worker processes that perform some computations and pass back the result to a controlling process. I installed zeromq.node and set up a simple requester and responder in coffeescript.

the requester:

# requester.coffee

zmq                       = require 'zmq'
context                   = new zmq.Context()
socket                    = zmq.socket 'req'

socket.bind 'tcp://127.0.0.1:5555', ( error ) =>
  throw error if error?
  console.log 'Requesting writer bound to port 5555'
  setInterval ( -> socket.send 'helo world' ), 1


response_count  = 0
t0              = new Date() / 1000

socket.on 'message', ( message ) ->
  response_count += 1
  # x = message.toString 'utf-8'
  if response_count % 1000 is 0
    t1              = new Date() / 1000
    console.log "received #{ parseInt response_count / ( t1 - t0 ) + 0.5 } messages per second"
    response_count  = 0
    t0 = new Date() / 1000

The responder:

# responder.coffee

zmq                       = require 'zmq'
context                   = new zmq.Context()
socket                    = zmq.socket 'rep'

socket.connect 'tcp://127.0.0.1:5555'
console.log 'Responder bound to port 5555'
process.stdin.resume()

request_count   = 0
t0              = new Date() / 1000

socket.on 'message', ( message ) ->
  request_count += 1
  # message = message.toString 'utf-8'
  # console.log message
  socket.send 'helo back'
  if request_count % 1000 is 0
    t1              = new Date() / 1000
    console.log "received #{ parseInt request_count / ( t1 - t0 ) + 0.5 } messages per second"
    request_count  = 0
    t0 = new Date() / 1000

Now when I run them in separate terminal windows on my ubuntu (11.10, 8GB, Intel Duo Core 3GHz, NodeJS 0.8.6) machine, I get the following output:

received 135 messages per second
received 6369 messages per second
received 6849 messages per second
received 6944 messages per second
received 7042 messages per second
received 7143 messages per second
received 5952 messages per second
received 2967 messages per second
received 914 messages per second
received 912 messages per second
received 928 messages per second
received 919 messages per second
received 947 messages per second
received 906 messages per second
received 918 messages per second
received 929 messages per second
received 916 messages per second
received 917 messages per second
received 916 messages per second
received 928 messages per second

which (1) looks a little like there is some kind of saturation in the transmission channel after a few seconds; (2) feels not fast enough. according to this benchmark, I should be in the hundreds of thousands – not thousands – of messages per seconds, which is corrobrated by this discussion ("ZeroMQ: It takes about 15 milli seconds to receive 10,000 messages").

I have also tried to to use a responder written in python 3 and got the exact same numbers. what's more, I wrote an alternative pair of scripts, where the master process would spawn a child process and communicate with it over stdout/stdin; I obtained around 750 messages per second (I couldn't see much variance when I increased message length), which is in like the same ballpark as the zeromq experiment.

Are these numbers to be expected? What is the limiting factor here?

Answer 1

I think two things are going on. First, you have setInterval ( -> socket.send 'helo world' ), 1. This sends a request every millisecond, so you are going to be limited to 1000 request per second.

Also, the request/response model is synchronous. A request comes in on a socket and blocks until a response is given. With the response, the next request is processed and blocked.

I changed the requester script from socket = zmq.socket 'req' to socket = zmq.socket 'push' and setInterval ( -> socket.send 'helo world' ), 0 to socket.send 'helo world' while true. I then changed the responder script socket = zmq.socket 'rep' to socket = zmq.socket 'pull' and got this output

$ coffee responder.coffee
Responder bound to port 5555
received 282 messages per second
received 333357 messages per second
received 249988 messages per second
received 333331 messages per second
received 250003 messages per second
received 333331 messages per second
received 333331 messages per second
received 333331 messages per second
...

The requester got no output because it was blocking on the while loop, but it does demonstrate that ØMQ can get much better performance.