Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a golang redis client that auto detects new shards for pubsub?

[UPDATE]: Current redis sends every published message to every node in entire cluster:

/* -----------------------------------------------------------------------------
 * CLUSTER Pub/Sub support
 *
 * For now we do very little, just propagating PUBLISH messages across the whole
 * cluster. In the future we'll try to get smarter and avoiding propagating those
 * messages to hosts without receives for a given channel.
 * -------------------------------------------------------------------------- */
void clusterPropagatePublish(robj *channel, robj *message) {
    clusterSendPublish(NULL, channel, message);
}

This is original text of question, which is not correct:

As I understand it I need to:

  1. Given a channel, find the the node that owns the hash slot.

  2. Subscribe to that node and also to cluster:slots to detect migrations.

  3. Upon slot migration, subscribe to the channel on the new node and keep the old connection open.

  4. Forward messages to the application from the old connection until it closes and remember those messages.

  5. When migration is complete and the old connection closed, forward messages from the new connection, trimming the remembered messages from the first connection.

Do any of the golang redis client library do this? I’ve looked through many and it feels like I need to write this logic myself, constantly polling CLUSTER SLOTS or listening to a pubsub of that info to know when shards increase or decrease and move my existing pubsub scriptions from one server to another.

I.e. there lots of golang libs that handle normal GET key with cluster even if the number of shards changes. But pubsub and cluster is a whole other thing right?

like image 956
Andrew Arrow Avatar asked Aug 28 '18 00:08

Andrew Arrow


1 Answers

PubSub channels in the Redis cluster are shared among all nodes - messages are passed via the internal bus so you don't need a special client and/or logic.

In a Redis Cluster clients can subscribe to every node, and can also publish to every other node. The cluster will make sure that published messages are forwarded as needed.

Source: Redis Cluster Specification

like image 127
Itamar Haber Avatar answered Oct 17 '22 02:10

Itamar Haber