Database connection pooling with multi-threaded service

Question

I have a .NET 4 C# service that is using the TPL libraries for threading. We recently switched it to also use connection pooling, since one connection was becoming a bottle neck for processing.

Previously, we were using a lock clause to control thread safety on the connection object. As work would back up, the queue would exist as tasks, and many threads (tasks) would be waiting on the lock clause. Now, in most scenarios, threads wait on database IO and work processes MUCH faster.

However, now that I'm using connection pooling, we have a new issue. Once the max number of connections is reached (100 default), if further connections are requested, there is a timeout (see Pooling info). When this happens, an exception is thrown saying "Connection request timed out".

All of my IDisposables are within using statements, and I am properly managing my connections. This scenario happens due to more work being requested than the pool can process (which is expected). I understand why this exception is thrown, and am aware of ways of handling it. A simple retry feels like a hack. I also realize that I can increase the timeout period via the connection string, however that doesn't feel like a solid solution. In the previous design (without pooling), work items would process because of the lock within the application.

What is a good way of handling this scenario to ensure that all work gets processed?

Answer 1

Another approach is to use a semaphore around the code that retrieves connections from the pool (and, hopefully, returns them). A sempahore is like a lock statement, except that it allows a configurable number of requestors at a time, not just one.

Something like this should do:

//Assuming mySemaphore is a semaphore instance, e.g. 
// public static Semaphore mySemaphore = new Semaphore(100,100);
try {
  mySemaphore.WaitOne(); // This will block until a slot is available.
  DosomeDatabaseLogic();
} finally {
  mySemaphore.Release();
}

Answer 2

You could look to control the degree of parallelism by using the Parallel.ForEach() method as follows:

var items = ; // your collection of work items
var parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = 100 };
Parallel.ForEach(items, parallelOptions, ProcessItem)

In this case I chose to set the degree to 100, but you can choose a value that makes sense for your current connection pool implementation.

This solution of course assumes that you have a collection of work items up front. If, however, you're creating new Tasks through some external mechanism such as incoming web requests the exception is actually a good thing. At that point I would suggest that you make use of concurrent Queue data structure where you can place the work items and pop them off as worker threads become available.