What is the magic behind Lightstreamer?

Question

I'm gonna develop a framework for comet programming, and I can't use Web Sockets, or Server-Sent Events (because browser support really sucks). So, I need to keep the HTTP connection alive, and send chunked data back to the client.

However, problems show themselves as you get into the work:

1. Using XMLHttpRequest is not possible, due to the fact that IE doesn't give you xhr.responseText while the xhr.readyState is 3.
2. A hidden iframe can't be useful, because browser shows the loader while I send data back to the client.
3. I tried to send a JavaScript file back to the client, sending function execution commands each time, but browsers won't execute JavaScript till it's completely loaded.

However, when I look at Lightstreamer demo page, I see that it sends a JavaScript file back to the client little by little and in each step, it sends a call to the function and that function simply gets executed (I can't do this part). It seems that Lightstreamer uses AJAX, since the request simply shows up in Firebug's console tab, but it works like a charm in IE too.

I tried to use every HTTP header field they've set on their request, and no result. I also tried to use HTTP Post instead of HTTP Get, but still got no result.

I've read almost over 20 articles on how to implement comet, but none of'em appear to solve problems I have:

1. How to make it cross-browser?
2. How to get notified when new data is arrived from server (what event should I hook into)?
3. How to make my page appear as completely loaded to the user (how to implement it, so that browser doesn't show loading activity)?

Can anyone please help? I think there should be a very little tip or trick that I don't know here to glue all the concepts together. Does anyone know what lightstreamer do to overcome these problems?

Answer 1

SockJS author here.

• How to make it cross-browser?

This is hard, expect to spend a few months on getting streaming transports on opera and IE.

• How to get notified when new data is arrived from server (what event should I hook into)?

There are various techniques, depending on a particular browser. For a good intro take a look at different fallback protocols supported by Socket.IO and SockJS.

• How to make my page appear as completely loaded to the user (how to implement it, so that browser doesn't show loading activity)?

Again, there are browser-specific tricks. One is to delay loading AJAX after onload event. Other is to bind-and-unbind an iframe from DOM. ETC. If you still feel interested read SockJS or Socket.io code.

Can anyone please help? I think there should be a very little tip or trick that I don't know here to glue all the concepts together. Does anyone know what lightstreamer do to overcome these problems?

Basically, unless you have a very strong reason to, don't reinvent the wheel. Use SockJS, Socket.io, faye, or any other of dozens projects that do solve this problem already.