Restlet streaming data

Question

I have this task that I'm undertaking where I would be reading data from a device and make it available over a web service. The data is read 4 times a second. I want the web clients to be have an open HTTP connection and get the device readings as a stream using chunked transfer as long as the client keeps the connection open.

As a proof of concept, I want to start with a service that constantly generates a random number, 4 times a second, wraps it in json and stream that to clients. I'm trying to model it loosely based on twitter streaming api.

I'm using restlet 2.1.2 to create that webservice but I'm not sure which Representation I should be using to achieve this. I tried searching for this but didn't find anything useful. Could someone point me in the right direction as to what I should be using and maybe some examples, perhaps.

Thanks

Answer 1

To achieve what you are trying to do, I'd use the WriterRepresentation (but see my answer to your other question), but I'm quite sure that you are going in the wrong architectural direction.

Indeed the following image from the documentation you linked

shows how even the Twitter streaming api is not intended to be connected by users, but by background processes that download messages in a store accessible by the HTTP. Users poll only the HTTP server, that reads the messages from the store and sends the back to the clients.

As a disconnected protocol, HTTP enable massive scalability that would not be possible otherwise. If each client establishes a persistent TCP connection backed by a dedicated server thread, you will rapidly exaust server resources! Moreover any HTTP proxy between the User Agent and the server could cause unexpected behaviours.

Thus, if you are bound to the HTTP protocol, the User Agent should poll. You can reduce the network load with headers like Last-Modified/If-Modified-Since or Etag/If-None-Match.

However, if you can adopt a different protocol, I strongly suggest to try a service bus over a connected TCP protocol.