Immediate and scheduled business transactions - MSMQ/NServiceBus/MS Sql Service Broker/Windows Task Scheduler?

Question

I have this scenario, guys-

A typical large-scale CMS application in which the following business transactions appear:

• Users publish content immediately.
• Users schedule content to be published.
• Automatic process runs ,say, in 10 minutes interval, collects scheduled and ready (simply a status of the BO)content and sends it to publishing.

This is oversimplified description of the actual system. By saying publishing I mean a complex process consisting of 4 main sub-systems: building invocation arguments, generating html content by running another web app(asp.net), committing the transactional parties, and signaling the users about the publishing results. So the system looses ability to run the whole process in a single Mega-transaction, i.e. it's possible but not practical from the scalability/performance points of view.

There are several options for the sub-systems to communicate each other, like making use of MSMQ, SQL Service Broker, NServiceBus, and a chain of plain WCF one-way invocations (which is currently implemented). So, guys, I'm looking for some possibly more reliable and scalable solution for this processing because looks like the system will be getting more busy with growing number of users and , therefore, more content to be generated. Especially, once, mobile versions' support is requested by the clients.

One of my considerations is to queue all the users immediate requests using MSMQ where a dedicated WCF service will dispatch them next to the content generation module (web app). And so will do the windows scheduled task. I can't see how these messaging platforms could be useful but not residing them before the html generation. This really confuses me.

Can't figure out what technology will fit best for this on my own.Any help, considerations, sharing your experience will help me, guys.

Answer 1

You could use the saga capabilities in NServiceBus to model the process of publishing, including the scheduling part, as it has the ability to perform actions reliably after a certain delay with practically no overhead.

If you're happy enough to queue users immediate requests, then you could easily use NServiceBus as your API instead of cobbling something together yourself with MSMQ, WCF, and Windows Scheduled Tasks.

Answer 2

If all the players are SQL Server instances (or backed by such) then Service Broker has a big advantage. As is integrated in the SQL Engine it means that all those times when you have to engage in a local distributed transaction between the database and the message store (ie. every time you enqueue or dequeue a message) become a ordinary simple transaction since the database is the message store. This also gives advantages when you consider backup/restore since you can have a coherent backup for both the data and the messages (the database backup contains a backup of all the messages), you only needs only one solution for high availability/disaster recoverability: the database solution (clustering, mirroring) is also the messaging HA/DR story since the database is the message store. And built in activation eliminates the need to configure (and more importantly, monitor and failover in HA/DR failover cases) the external activation mechanisms. Not to mention that built-in activation has no latency while it can also adapt to spikes, something external scheduled activation can have problems achieving (external tasks must pool and have to balance the frequency of pooling with desired latency and consider spikes)