An ASP.NET 3.5 webapp has to start several tasks that takes hours to complete. For obvious reasons the pages which starts these tasks cannot wait for them to finish nor will anyone want to wait that long to get a response, so the tasks must be asynchronous.
There is a Helper class to handle all of these long running tasks. The main method that schedules and executes these tasks is currently the following:
public static bool ScheduleTask(TaskDescriptor task, Action action)
{
bool notAlreadyRunning = TasksAsync.TryAdd(task);
if (notAlreadyRunning)
{
Thread worker = null;
worker = new Thread(() =>
{
try { action(); }
catch(Exception e)
{
Log.LogException(e, "Worker");
}
TasksAsync.RemoveTask(task);
workers.Remove(worker);
});
workers.Add(worker);
worker.Start();
}
return notAlreadyRunning;
}
On earlier implementations we've used the ThreadPool.QueueUserWorkItem
approach but the result has always been the same: after aprox. 20-30 mins a Thread was being aborted exception is thrown.
Does anyone know why is this happening? or how can it be prevented?
More Info:
UPDATE: Decisions
Thank you all for your responses. Now I don´t know which question to mark as answer. All of them are valid and are possible solutions to this problem. Will wait for today and mark as answer the answer with the most up votes, in case of a draw I will choose the first shown answer, typically they are ordered by most relevance.
For anyone who want´s to know the solution I choose, again due to time restrictions, was to change the IIS recycling configuration, But what I consider to be the ideal solution, based on my research and of course the answers below, is to create a "Worker Service" and use a communication solution between the ASP.NET App and the new "Worker Service" to coordinate the long running work to be done.
You can start the long-running process in its own application domain.
In the past, when I've needed this capability, I create a Windows Service for this purpose. If you use WCF to connect to it, it doesn't even have to run on the IIS machine at all; you can run it on any machine on the network.
Chances are you can get this working, by upping the timeout, using a different app pool or a variety of other hacks, but your best bet is going to be to decouple the long running task from the ui and asp.net completely, and use either a service (wouldn't recommend it) or a scheduled task that polls for work to do; personally I would use something like aws sqs/sns to keep track of work to be done and a scheduled task in windows server that checks for things todo at whatever frequency make sense. The only thing the ui/asp.net then needs to do is log that fact that something needs to be done, not actually do it.
Another benefit of this message based approach is should the long running process become so long running, or so overworked, you'd have the opportunity to add more worker tasks or servers to complete those requests.
Perhaps more than you can implement for your immediate problem, but something to consider for a better long term solution.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With