Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Scalable Pub/Sub engine for realtime

I'm looking for a pub/sub engine, with the following requirements:

  1. Very low latency < 0.5 sec
  2. Scalable
  3. Shardable (based on geo localisation)

I'd like to be able to have multiple pub/sub servers and be able to publish or subscribe to channels from any server, no matter on which server the channel is declared.

For example:

If user A is connected to server SRV1 and user B connected to server SRV2, If user B subscribe to "MyChannel" and user A publish something on channel "MyChannel", user B will get the message even if he's not connected to the same server.

I don't know if Redis is able to do that. I didn't find anything about the subject.

like image 646
Vinc Avatar asked Sep 19 '11 23:09

Vinc


People also ask

Is Pubsub real time?

Pub/Sub: notify your clients in real time This channel is represented by a persistent network communication (WebSocket, MQTT, etc.). Customers, or subscribers, will first subscribe to notifications received in a specific channel. Then the server, or publisher, will send data to the clients through this channel.

Is Pubsub scalable?

Pub/Sub is an asynchronous messaging service designed to be highly reliable and scalable. The service is built on a core Google infrastructure component that many Google products have relied upon for over a decade.

How reliable is Redis pub sub?

The Redis pub/sub is not a reliable messaging system. Messages that are not retrieved will be discarded when your client is disconnected or a master/standby switchover occurs.

When should you not use a pub sub?

There are scenarios where Pub/Sub is the wrong choice: Overkill for simpler systems: Pub/Sub is an overkill for simple system which are unlikely to scale up. If you're operating systems where elastic scaling is not required and where static scaling will suffice, think twice before using Pub/Sub.


2 Answers

We've been using ZeroMQ and it's pub/sub features for while now and we're very happy with what we're seeing.

It's also worth looking at what's coming up in the next version (reducing network bandwidth by pushing subscription requests upstream)

like image 102
Alan Avatar answered Nov 01 '22 00:11

Alan


I suggest you look at Data Distribution Service for Real Time Systems (DDS) standard. It's specifically designed to be a scalable pub/sub middleware both for real-time and non-real-time systems.

It has a few mature implementations all of which has it's own strength points, but generally the implementations are scalable and low latency.

These are the implementations I would suggest you look at (If you need them to work on a WAN environment, I guess the first two ones have great support for that):

  • RTI DDS
  • OpenSplice DDS
  • OpenDDS
  • CoreDX DDS
like image 31
Ertan D. Avatar answered Nov 01 '22 00:11

Ertan D.