Can't receive redis data from socket io

Question

I'm building a realtime visualization using redis as pubsub messenger between python and node. There's a python script always running which sets a redis hash with hmset. That side of the app is working fine, if I enter the following example command: "HGETALL 'sellers-80183917'" in a redis client I end up getting the proper data.

The problem is in the js side. I'm using socketio and redis nodejs libraries to listen to the redis instance and publish the results online through a d3js viz.

I run the following code with node:

var express = require('express');
var app = express();
var redis = require('redis');

app.use(express.static(__dirname + '/public'));


var http = require('http').Server(app);


var io = require('socket.io')(http);
var sredis = require('socket.io-redis');
io.adapter(sredis({ host: 'localhost', port: 6379 }));



redisSubscriber = redis.createClient(6379, 'localhost', {});


redisSubscriber.on('message', function(channel, message) {
  io.emit(channel, message);
});



app.get('/sellers/:seller_id', function(req, res){
  var seller_id = req.params.seller_id;
  redisSubscriber.subscribe('sellers-'.concat(seller_id)); 


  res.render( 'seller.ejs', { seller:seller_id } );
});


http.listen(3000, '127.0.0.1', function(){
  console.log('listening on *:3000');
});

And this is the relevant part of the seller.ejs file that's receiving the user requests and outputting the viz:

        var socket = io('http://localhost:3000');
        var stats;
        var seller_key = 'sellers-'.concat(<%= seller %>);
        socket.on(seller_key, function(msg){
            stats = [];
            console.log('Im in');
            var seller = $.parseJSON(msg);
            var items = seller['items'];
            for(item in items) {
                var item_data = items[item];
                stats.push({'title': item_data['title'], 'today_visits': item_data['today_visits'], 'sold_today': item_data['sold_today'], 'conversion_rate': item_data['conversion_rate']});
            }
            setupData(stats);
        });

The problem is that the socket_on() method never receives anything and I don't see where the problem is as everything seems to be working fine besides this.

Answer 1

I think that you might be confused as to what Pub/Sub in Redis actually is. It's not a way to listen to changes on hashes; you can have a Pub/Sub channel called sellers-1, and you can have a hash with the key sellers-1, but those are unrelated to each other.

As documented here:

Pub/Sub has no relation to the key space.

There is a thing called keyspace notifications that can be used to listen to changes in the key space (through Pub/Sub channels); however, this feature isn't enabled by default because it'll take up more resources.

Perhaps an easier method would be to publish a message after the HMSET, so any subscribers would know that the hash got changed (they would then retrieve the hash contents themselves, or the published message would contain the relevant data).

This brings us to the next possible issue: you only have one subscriber connection, redisSubscriber.

From what I understand from the Node.js Redis driver, calling .subscribe() on such a connection would remove any previous subscriptions in favor of the new one. So if you were previously subscribed to the sellers-1 channel and subscribe to sellers-2, you wouldn't be receiving messages from the sellers-1 channel anymore.

You can listen on multiple channels by either passing an array of channels, or by passing them as a arguments:

redisSubscriber.subscribe([ 'sellers-1', 'sellers-2', ... ])
// Or:
redisSubscriber.subscribe('sellers-1', 'sellers-2', ... )

You would obviously have to track each "active" seller subscription. Either that, or create a new connection for each subscription, which also isn't ideal.

It's probably a better idea to have a single Pub/Sub channel on which all changes would get published, instead of a separate channel for each seller.

Finally: if your seller id's aren't hard to guess (for instance, if it's based on an incremental integer value), it would be trivial for someone to write a client that would make it possible to listen in on any seller channel they'd like. It might not be a problem, but it is something to be aware of.