Maximum queued elements in ThreadPool.QueueUserWorkItem

Question

I set the max thread to 10. Then I added 22000 task using ThreadPool.QueueUserWorkItem. It is very likely that not all the 22000 task was completed after running the program. Is there a limitation how many task can be queued for avaiable threads?

Answer 1

If you need to wait for all of the tasks to process, you need to handle that yourself. The ThreadPool threads are all background threads, and will not keep the application alive.

This is a relatively clean way to handle this type of situation:

 using (var mre = new ManualResetEvent(false))
 {
      int remainingToProcess = workItems.Count(); // Assuming workItems is a collection of "tasks"
      foreach(var item in workItems)
      {
           // Delegate closure (in C# 4 and earlier) below will 
           // capture a reference to 'item', resulting in
           // the incorrect item sent to ProcessTask each iteration.  Use a local copy
           // of the 'item' variable instead.
           // C# 5/VS2012 will not require the local here.
           var localItem = item;
           ThreadPool.QueueUserWorkItem(delegate
           {
               // Replace this with your "work"
               ProcessTask(localItem);

               // This will (safely) decrement the remaining count, and allow the main thread to continue when we're done
               if (Interlocked.Decrement(ref remainingToProcess) == 0)
                      mre.Set();
           });
      }
      mre.WaitOne();
 }

---

That being said, it's usually better to "group" together your work items if you have thousands of them, and not treat them as separate Work Items for the threadpool. This is some overhead involved in managing the list of items, and since you won't be able to process 22000 at a time, you're better off grouping these into blocks. Having single work items each process 50 or so will probably help your overall throughput quite a bit...