Building a SignalR / Knockout dashboard with guaranteed messaging

Question

I'm looking into replacing a monitoring dashboard at our company using real-time messaging.

The Old Concept:

At our company we have a dashboard that displays the (rather detailed) status of over 700 physical machines, plus added meta-information. It was built about 1,5 years ago by a colleague of mine in ASP.NET Web Forms (which I don't like) to enable dispatchers to coordinate where our technicians should go to fix problems (the machines are located in various geographical locations).

Unfortunately, the app uses a 30-second complete page auto-refresh with a big query behind it. It's slow and it completely resets your view (as I said, the dashboard contains over 700+ machines). Personally I would like to change this. It is extremely annoying to use. Our dispatchers have learned to live with this, but I think they deserve better.

The New Concept:

I want to display the same content on a new dashboard, but with real-time updates and "message" log. At our company we work for about 90% on a MS stack, so I plan on using ASP.NET MVC, SignalR, SQL Server and Knockout.

What I currently have

Take a look at this simple diagram:

 +----+ +----+ +----+ +----+ +----+ +----+ +----+            
 | PC | | PC | | PC | | PC | | PC | | PC | | PC | ... ...    
 +--+-+ +--+-+ +-+--+ +--+-+ +--+-+ +--+-+ +--+-+            
    |      |     |       |      |      |      |              
    |   +--+  +--+  +----+    <-+    <-+    <-+              
    |   |     |     |                                        
+---v---v-----v-----v+         +-----------------------+     
|                    | TCP/IP  |                       |     
| Monitoring Backend +--------->  Data Enrichment App  |     
|                    |         |                       |     
+--------------------+         +---------+-------------+     
                                         |                   
          +------------------------------+        +---------+
          |                                       |         |
          |                +----------------+     |         |
    +-----v-----+---------->    DB Proxy    +----->  S Q L  |
    |           | PUB/SUB  +----------------+     |         |
    |   Redis   |                                 |         |
    |           |          +----------------+     +---------+
    +-----------+          |     TO BE...   |                
                           +----------------+                

• I created a little "Data enrichment app" that receives events from the monitoring backend over TCP/IP and adds additional business data to the event (For example, the location of the devices, a descriptive name alongside the hostname, a humn-readable translation of the alert, etc.) that is not contained in the monitoring system.
• Enriched events are sent from the app to Redis. I did this so other applications can hook into Redis as a subscriber, since the data I'm outputting here is superior and more readable than what the monitoring backend sends out.
• Currently the only thing PUB/SUB-ing to Redis is a DB proxy that listens for incoming events and sends those to the Database (SQL Server), which I already use for historical reporting purposes, but currently only contains rather simplistic data.

The idea here is to subscribe a SignalR Hub to the Redis backend in my ASP.NET application to fire events over to the client. (That's the TO BE part)

The Problem:

The idea is that when the client navigates to the dashboard URL, the initial overview is populated by the status data that's in the SQL backend. Afterwards, events are received through SignalR and the view is updated by changing Knockout properties.

However, should a client get disconnected (say, by sleeping his laptop when walking from meeting room to meeting room) he misses messages from the SignalR hub, and his dashboard view is no longer correct!

Possible solutions would be:

1. Sending the complete status of every device through SignalR on every event change: This is impossible because of the huge amount of data I would have to send over the wire. (I'm guessing at least 12,000 records of JSON data)

2. Forcing a complete refresh after detecting a timed-out connection: I have no idea how to implement this using SignalR :(

3. ... ?

What is the recommended approach to dealing with Real-time, push-based data and guaranteeing that data arrives? Or how would I deal with recovering from a timed-out connections? Or is the idea of making this real-time crazy?

Disclaimer: I'm a system engineer, not a profession programmer. This is my first real-time web app. Other questions regarding SignalR usually don't deal with large amounts of data like this.

Answer 1

spender's answer is good, but I'd like to address solution 2 in the context of SignalR; you could use the SignalR lifetime events for this: OnConnected, OnReconnected and OnDisconnected. You can read more about the events here and how to use them in a hub here.

You'd fully initialize the view when the client first connects (OnConnected gets called). If a client loses connection temporarily (by default less than 30s, see relevant settings here, OnReconnected gets called), you don't need to do anything else; the queued messages will be delivered as long as there is enough space in the standard queuing mechanism.

If the client PC goes to sleep, OnDisconnected gets called eventually and the client will have to establish a new connection. At that point, the easiest implementation would be to simply load all data again. If you want to reuse the (outdated) data the client already has, then you'd need

• a way to retrieve a subset of data / messages (e.g. based on a seq number or a timestamp; Since you're already storing the event stream to a DB, it sounds like it should be possible to integrate this)
• store this number on the client whenever it receives a message
• send it to the server when establishing the connection (e.g. via the query string), so that the server can read it in OnConnected and knows whether to initialize a full view or only a changeset

Using SignalR messaging for the real-time updates should be fine, however I'd suggest using a regular MVC / WebAPI controller to serve the full dataset necessary to initialize the view (from OnConnected).

That said, if you want guaranteed delivery, you'll have to ack your messages and probably also implement a queueing mechanism. SignalR only buffers about 1000 messages by default, then it starts dropping them. You can increase that value, but it may make more sense to build one tailored to your requirements.

Answer 2

SignalR is not designed for "Reliable Messaging". It was designed for "Real Time Communication".

The issue is that Reliability is in fact incompatible with Real Time. Reliable messaging means that a message will be delivered at least once. However if the data link is down, then the message will be delivered delayed. However Real Time means that the message is delivered "instantly", not half an hour later.

I would switch to a "Message Queue" if reliability is what you need. You should find that they are "quick enough" for your purposes (RTC typically means you need latency in the millisecond range, whilst MQ should give typical latency in the second range).

What your question is asking is, how do I implement a Message Queue over SignalR.

Try RabbitMQ, I've heard only good things with it. There is a Javascript client as well.