Max tasks in TPL?

Question

I want to use TPL in Worker process on Windows Azure. I'm looking to add an IJob the queue, this has a Run method, so the worker will consist of:

loop get item off queue Use TPL to call IJob.Run, this is an async call

But I'm a bit concerned about the maximum items I can add to TPL? I'm happy to build my own TPL Pool of some sort if required, just checking it capabilities.

Cheers, Ash.

Answer 1

One of the main goals of the TPL is to remove the need to worry about this. By decomposing your work into Tasks instead of Threads, you're allowing the scheduler to handle the balancing of this more appropriately.

There is no fixed upper limit to the number of "tasks" you can schedule. They are (by default, with the default TaskScheduler) scheduled using the ThreadPool, which as of .NET 4, scales based on the work. I would strongly suggest not trying to build your own pool - it's highly unlikely that you'll do better than the default. That being said, if your tasks have a very non-standard behavior, you may want to consider writing a custom TaskScheduler.

Also - realize that you should, ideally, make your tasks as "large as possible". There is overhead associated with an individual task - having them be too small (in terms of work) will cause the overhead to have a larger impact on performance than if you have an appropriate number of larger "tasks".