run methods from all users in a queue

Question

My setup: User has a page where he performs some actions and data transformations.
I store those actions in a dataset (in a session).
Then, on a button click I need to perform some more transformation and call custom database function that will insert each row from each datatable into a database. I have all that worked out and it works pretty good.

However, now, what I need to do is to modify a button click event not to run immediately, but put that action in a queue (it's a common queue for all users). So if user1 click the button, then user2 clicked the button, then we need to take user1 dataset and run method on it, and only after that take user2 dataset and run method on it. one queue for all.

private void btnclick()
{
  DataSet ds = Session["test"] as DataSet;
  PerformFinalTransformation(ds);
  foreach (DataTable dt in ds.Tables)
  {
     foreach (DataRow dr in dt.Rows)
     {
        CallCustomSqlFunction(dr);
     }
  }
}

I'd like to avoid storing datasets in a database if possible, since they are not the same for each user (number of tables/columns). I'm looking for any pointers on how this can be done in asp.net /c#. (Is quartz.net something I should consider?). What concepts should I look for? parallel programming? or asp.net queue? or something else? Cannot even start, since do not know what to start with. Btw, I use the latest DotNetNuke in my app if that helps somehow.

Answer 1

From an architectural standpoint, your solution is good. A queue is the way to go. However, the solution is non-trivial. Some stuff to consider:

1. Resilience: what if the system crashes or restarts? Do the queue is preserved?
2. How will you consume from this queue? A thread in your ASP.Net application (ugly)? A Windows Service?
3. What about concurrency?

Depending on your answers, I give you some alternatives:

1. A queue in memory, using a [ConcurrentQueue][1], that goes up in a thread. Probably using a Singleton to enable access to the queue. I DO NOT ADVISE YOU TO DO THIS.

2. Create a service, expose interfaces through WCF and access it from you ASP.Net application. The service should hold a [ConcurrentQueue][2]as well, and it will be the service responsibility to access the database, etc. (you still will have problems with crashes and data loss).

3. Use MSMQ. And some services to encapsulate functionality to receive calls from ASP.Net (if you don't want to have ASP.Net access MSMQ directly) and to access the database (MSMQ calls this service).

These are just from the top of my head. Please feel free to comment and I'll answer as I can (I'm currently working).