Node.js and Comet

Question

Basically, I am trying to set up a basic comet server and client using node.js from scratch. A tried to Google things about the process it should work and as I am testing it seems to work fine. However one question is still in my head. First I would like to show you the code.

var http = require('http');
var sys = require('sys');
var fs = require('fs');
var qs = require('querystring');

var server = http.createServer();

var connections = [];

server.on('request', function(req, res) {

    console.log(req.url);

    if(req.url == '/index') {

        fs.readFile(__dirname + '/index.html', function(err, data){
            if(err) {
                res.writeHead(err, data);
                return res.end('Error loading index.html');
            }

            res.writeHead(200);
            res.end(data);
        });
    } else if(req.url == '/sendData') {
        if (req.method == 'POST') {
            var body = '';
            req.on('data', function (data) {
                body += data;
            });
            req.on('end', function () {

                var POST = qs.parse(body);
                console.log(POST);
                res.writeHead(200);
                res.end("");
                broadcastData(POST);
            });
        }

    } else {
        connections.push(res);
    }
});

function broadcastData(data) {
    for(var i = 0; i < connections.length; i++) {
        connections[i].writeHead(200);
        connections[i].end(JSON.stringify({'message': data}));
    }
}

process.openStdin().addListener('data', function(chunk) {
    for(var i = 0; i < connections.length; i++) {
        connections[i].writeHead(200);
        var message = chunk.toString();
        connections[i].end(JSON.stringify({'message': {'name': message}}));
    }
});

server.listen(4000);

the client side:

function doComet() {
    $.getJSON('/', function(events){
        doComet();
        console.log(events);
        $('#content').append(events.message.name);
    });
}

function sendData(data) {
    $.ajax({
        type: "POST",
        url: "/sendData",
        contentType: 'application/javascript; charset=UTF-8',
        data: { name: "John", location: "Boston" }
    }).done(function( msg ) {
        console.log(msg)
    });
}

So there is a listener which sends the data which I write on stdin and also there's a button on the client side which will send the data to the server and the server sends it to all clients.

My question is what if a lot of events happen in a really short amount of time? I mean as soon as a response got from the server another ajax request is sent but there should be a short amount of time when the client is not connected yet and if happens something during that time the client will not have the fresh data.

So does my thoughts right about this? In this case somehow I should sync the data properly to make sure everybody gets it. Does anybody know how to do it properly?

Thank you for you help!

Answer 1

Comet is an umbrella term for all the old HTTP-based hacks and is a phrase we want to get rid of. WebSockets are where we want to be; browsers and beyond. So, if you are interesting in building a realtime solution then you should probably first investigate WebSockets and then deal with fallbacks such as HTTP Streaming and HTTP Long-Polling. For more information see Realtime Web Technology Transport Mechanisms.

The example you've provided would be classed as HTTP Long-polling.

what if a lot of events happen in a really short amount of time? I mean as soon as a response got from the server another ajax request is sent but there should be a short amount of time when the client is not connected yet and if happens something during that time the client will not have the fresh data

This is one of the limitations with HTTP Long-polling and why HTTP Streaming is a better solution, and WebSockets are better still - the connection isn't closed when data is sent. With any kind of polling solution you're probably going to need a solution which ensures that messages received, and sent, during the polling requests aren't missed by the polling client.

There are a couple of solutions:

1. Use sessions to keep track of the last message that was sent to a particular client. When they reconnect check which messages was last send, which messages have been missed and send.
2. As part of the client polling send a lastMessageId. When the server receives this you can check to see if that is the last message. If not you can respond with any missed messages.

One of the complexities with realtime web technologies is handling all the possible connection scenarios. So, if you are doing this as a learning experience then it's definitely beneficial, and fun. But if you are creating an application to move into production I would recommend an existing realtime framework. See the realtime web tech guide.