ThreadPool frustrations - Thread creation exceeding SetMaxThreads

Question

• I've got an I/O intensive operation.
• I only want a MAX of 5 threads ever running at one time.
• I've got 8000 tasks to queue and complete.
• Each task takes approximately 15-20seconds to execute.

I've looked around at ThreadPool, but

        ThreadPool.SetMaxThreads(5, 0);

        List<task> tasks = GetTasks();

        int toProcess = tasks.Count;
        ManualResetEvent resetEvent = new ManualResetEvent(false);

        for (int i = 0; i < tasks.Count; i++)
        {
            ReportGenerator worker = new ReportGenerator(tasks[i].Code, id);
            ThreadPool.QueueUserWorkItem(x =>
            {
                worker.Go();
                if (Interlocked.Decrement(ref toProcess) == 0)
                    resetEvent.Set();
            });
        }

        resetEvent.WaitOne();

I cannot figure out why... my code is executing more than 5 threads at one time. I've tried to setmaxthreads, setminthreads, but it keeps executing more than 5 threads.

What is happening? What am I missing? Should I be doing this in another way?

Thanks

Answer 1

Task Parallel Library can help you:

List<task> tasks = GetTasks();

Parallel.ForEach(tasks, new ParallelOptions { MaxDegreeOfParallelism = 5 }, 
  task => {ReportGenerator worker = new ReportGenerator(task.Code, id); 
           worker.Go();});

What does MaxDegreeOfParallelism do?