Message queues in ASP.Net Web API

Question

I am developing a client-side single-page-application (SPA) with AngularJS and ASP.Net WebAPI.

One of the features of the SPA includes uploading large CSV file, processing it on the server, and returning the output to the user.

Obviously, this kind of computation can not be done online, and therefore I implemented an UploadController in charge of receiving the file, and a PollingController in charge of notifying the user when the computation is complete.

The client side application monitors the PollingController every few seconds.

I have no experience in Message Queues, but my gut tells me that they are required in this situation.

How would you recommend to implement this functionality in a non-blocking, efficient way ?

Examples will be highly appreciated

Answer 1

I've used message based service bus frameworks for this in the past.

You write an application (running as a windows service), that listens for messages broadcast across a event bus.

Your frontend can publish these messages into the bus.

The most popular framework for this in .NET is NServiceBus, however it recently became commercial. You can also look into MassTransit, though this one has very poor documentation.

The workflow you would do:

• MVC App accepts upload and places it into some directory accessible by the windows service
• MVC App publishes "UploadReady" message.
• Service receives message, processes file, and updates some state for the polling controller.
• Polling controller watches for this state to change. Usually a DB record etc.

The nice bit about using a framework like this is that if your service goes down, or you redeploy it, any processing can queue and resume, so you won't have any downtime.