TCP/IP communications with NodeJS for multiple write messages

Question

I am working on building a distributed caching system using python over TCP/IP and I have implemented the TCP SERVER using python and I am creating TCP client libraries for Python, PHP and NodeJS.

So the client will request for a cache item using a and the TCP server will return the cache value back to the client

I am facing an issue with NodeJS since it is an Asynchronous event based programming tool.

Below is the pseudo code for PHP client library

$server = new CacheServer();
$server->connect("<host>", "<port>");
$value_1 = $server->get("<cache_key_1>")
$value_2 = $server->get("<cache_key_2>")

so what the $server->get method does is first writes a message on the socket e.g. "GET <cache_key_1>" and then listen on the socket for the server to respond and then return the value for that cache key and only then move on to <cache_key_2>

But for NodeJS since it was based on event, Both the request for <cache_key_1> and <cache_key_2> are sent without waiting for response from the server.

The sockets reads both the values from server back but I am not able to differentiate which value is for which request. I know TCP socket requests are queued up so may be I have to track which request was sent first like in an array and have another array of responses and then map those.

Is there a easier way which I am not able to see?

In summary I want to send multiple write message from the TCP client using NodeJS and wait for response back from server between each "write" message

Any help will be greatly appreciated Thanks!

Answer 1

Idea

When dealing with async using sockets, the idea would be to tag the request with an identifier, and register a handler for that identifier, and when response is received, we match with registered identifiers. This is because we do not conceptualize sockets as request/response.

Here is a sample reservoir module (reservoir.js):

var net = require('net');

module.exports = {
    client: {},
    connect: function(port, host, then) {
        var $this = this;
        var client = net.Socket();

        client.connect(port, host, function(){
            then && then();
        });

        client.on("data", function(data) {
            $this._process(data);
        });

        this.client = client;
        return this;
    },
    get: function(key, then) {
        var $this = this;
        $this.client.write(key, function() {
            $this._handle(key, then);
        });
        return this;
    },
    _handlers: {},
    _commands: ['GET', 'SET', 'DEL'],
    _handle: function(key, handler) {
        //remove commands from the key, because server doesn't return them (you can check here if its a valid command)
        var keyParts = key.split(' ');
        keyParts.shift();

        this._handlers[keyParts] = handler;
    },
    _process: function(data) {
        var response = JSON.parse(data.toString());
        var handler = this._handlers[response.incoming_message];
        if (handler) {
            if (response.message) {
                handler(null, response.message);
            } else {
                handler(new Error(response.error), null);
            }
        }
        delete this._handlers[response.incoming_message];
    }
};

Here we are registering each callback with the associated key - so when we receive the data, we fire up the specific handler.

After all the abstraction in the module file, here is how it can be used:

var reservoir = require('reservoir');
var onConnect = function() {
    console.log("\nConnected.");
    reservoir.get('GET pk_movie', function(err, response) {
        console.log(response);
    });
}
reservoir.connect('3000', 'your-host.com', onConnect);

Hope that helps!