Node.js - Monitoring a database for changes

Question

I am using a node.js server to create a 'close to real-time' socket between my web app and a database. Currently I am using MySQL which I am polling every second in node to check if there are any changes to the table (based on a timestamp.)

I was wondering if there any specific techniques to doing something like this with MySQL? At the moment, I am just running a SQL query and using setTimeout before the next poll.

I know it's more common to use a NoSQL database in instances like this but I'm not really comfortable with the technology and I'd much rather use SQL.

Does anyone have any experience or tips for monitoring a SQL database with node?

Answer 1

I wouldn't personally use a polling mechanism for this. I think this is a pretty good use case for a pub/sub mq as a component on top of the database that allows consumers to subscribe to specific channels for change events on entities that they care about.

Ex:

1. Someone requests that a model be changed
2. Model broadcasts change event
3. Queue up change to be persisted in the database
4. Fire off a change set on a specific channel in a message queue for distribution to all interested parties

You can use a very simple in process pub/sub mechanism for this type of thing using nodes EventEmitter, and as you need to scale, have durability requirements, or need a cross language MQ you can go to a technology like rabbitmq, zeromq, etc. I've started to implement something very lightweight to do just this in one of my applications: https://github.com/jmoyers/mettle/blob/master/src/pubsub.coffee

It boils down to something like:

pubsub.sub('users.*', function(updates){
    // Interested party handles updates for user objects
});

That way you aren't putting stupid polling pressure on your database. In fact, change distribution is completely independent of writing to your database

Josh