Architecture query.. Building a service/message bus with Node.js

Question

So the situation is that I have a variety of datasources that are providing a stream of messages from external devices.. some are sending messages on a serial port, some via UDP, some via Telnet.. I wish to build a small Node.js system that receives messages from each of these sources. Around 20 sources all up.

I have a variety of places that I then want to relay these messages out to, and I wish to allow clients to connect via telnet and receive these messages.

I guess you could call it a "service bus" or a "message bus"..

At the moment I'm just kicking around ideas for how to structure it, I dont want one huge node.js file that does everything..

I want each of the "receivers" to be in external .js files to my main node.js file.. Is my approach below OK and are there any improvements I could make to my approach.

Main Node.js file

Pulls in two "receivers".. each of these will process incoming messages from a datasource

var sys = require("sys");

AVLReceiver = require("./avlreceiver.js").AVLReceiver();
PagerReceiver = require("./pagerreceiver.js").PagerReceiver();

pagerreceiver.js - a sample message receiver

Receives messages from a serial port..

var serialport = require("serialport");
var sys = require("sys");

var PagerReceiver = {};

PagerReceiver.initializePort = function () {
    this.serialport = new serialport.SerialPort("/dev/ttyS0", { 
        parser: serialport.parsers.readline("\n"), baudrate: 57600 
      });

    this.serialport.on("data", this.processMessage);
};

PagerReceiver.processMessage = function (data) {
  //deal with the message
};

PagerReceiver.initializePort();

exports.PagerReceiver = function() {
       return PagerReceiver;
};

Would this be an appropriate way to break up a node.js system? Any comments on the javascript also gratefully received.. Also any comments on any other architectural approaches I should consider for building a message bus in node.js would be super.

Thanks for reading,

Duncan.

Answer 1

This post is over 8 years old and the problem would be well and truly solved; but I thought I'd drop in some ideas using modern Node and Typescript for anyone who comes by this way.

A message bus is a really good fit as you found out. This helps to not overwhelm your application when the devices start sending out large volumes of messages.

A clean way to approach this would be to use a dedicated service bus like @node-ts/bus that deals with all the technical complexities of retrying messages, subscribing to topics etc.

The next would be to build in an Anti-corruption layer for the messages generated by the devices. When each message is received by them, it's translated into a domain message that conforms to your standards. This will prevent each message handler from having to have multiple concerns around deciphering messages and actioning them.

Answer 2

This is an older question so you've probably built your solution already, but I'll add my take on it just in case it's useful to somebody.

The idea of keeping your receiver-specific code isolated definitely seems right to me, it'll make it clear to the reader which code relates to the main workflow and which code relates to a specific stream.

I'd also be tempted to try;

• If possible giving all your Receivers the same interface so that the main code is simple.
• Maybe use EventEmitters to trigger events that can be caught by higher-level app code
• Externalise config such as serial ports and baud rates to a per-app or per-environment config file. Node's module system automatically checks "~/node_modules" so you can put per-environment config in there to avoid overwriting when you deploy new code.
• I'd be tempted to take a look at the way your module exports its functionality, see below.

Instantiation

I find the way you're tackling the object creation slightly misleading, as it makes exports.PagerReceiver() seem like a class constructor, which it isn't; in this case it's returning a singleton object. The module itself is already a singleton instance so this is a bit redundant and potentially misleading.

Below is an example of how multiple calls to require() actually reference the same private variable i.

counter.js

var i = 0;

exports.iterate = function(){
    return i++;
};

test.js

var counter1 = require('./counter');
var counter2 = require('./counter');
console.log(counter1.iterate());
console.log(counter2.iterate());
console.log(counter1.iterate());
console.log(counter2.iterate());

output:

0
1
2
3

The following code is simpler and functionally the same, aside from being called with require() instead of require().PagerReceiver():

var serialport = require("serialport");
var sys = require("sys");

exports.processMessage = function (data) {
    //deal with the message
};

var port = new serialport.SerialPort("/dev/ttyS0", { 
    parser: serialport.parsers.readline("\n"), baudrate: 57600 
});

port.on("data", exports.processMessage);