"Fixed" / "Load Balanced" C# thread pool?

Question

I have a 3rd party component that is "expensive" to spin up. This component is not thread safe. Said component is hosted inside of a WCF service (for now), so... every time a call comes into the service I have to new up the component.

What I'd like to do instead is have a pool of say 16 threads that each spin up their own copy of the component and have a mechanism to call the method and have it distributed to one of the 16 threads and have the value returned.

So something simple like:

var response = threadPool.CallMethod(param1, param2);

Its fine for the call to block until it gets a response as I need the response to proceed.

Any suggestions? Maybe I'm overthinking it and a ConcurrentQueue that is serviced by 16 threads would do the job, but now sure how the method return value would get returned to the caller?

Answer 1

WCF will already use the thread pool to manage its resources so if you add a layer of thread management on top of that it is only going to go badly. Avoid doing that if possible as you will get contention on your service calls.

What I would do in your situation is just use a single ThreadLocal or thread static that would get initialized with your expensive object once. Thereafter it would be available to the thread pool thread.

That is assuming that your object is fine on an MTA thread; I'm guessing it is from your post since it sounds like things are current working, but just slow.

There is the concern that too many objects get created and you use too much memory as the pool grows too large. However, see if this is the case in practice before doing anything else. This is a very simple strategy to implement so easy to trial. Only get more complex if you really need to.

Answer 2

First and foremost, I agree with @briantyler: ThreadLocal<T> or thread static fields is probably what you want. You should go with that as a starting point and consider other options if it doesn't meet your needs.

A complicated but flexible alternative is a singleton object pool. In its most simple form your pool type will look like this:

public sealed class ObjectPool<T>
{
    private readonly ConcurrentQueue<T> __objects = new ConcurrentQueue<T>();
    private readonly Func<T> __factory;

    public ObjectPool(Func<T> factory)
    {
        __factory = factory;
    }

    public T Get()
    {
        T obj;
        return __objects.TryDequeue(out obj) ? obj : __factory();
    }

    public void Return(T obj)
    {
        __objects.Enqueue(obj);
    }
}

This doesn't seem awfully useful if you're thinking of type T in terms of primitive classes or structs (i.e. ObjectPool<MyComponent>), as the pool does not have any threading controls built in. But you can substitute your type T for a Lazy<T> or Task<T> monad, and get exactly what you want.

Pool initialisation:

Func<Task<MyComponent>> factory = () => Task.Run(() => new MyComponent());
ObjectPool<Task<MyComponent>> pool = new ObjectPool<Task<MyComponent>>(factory);

// "Pre-warm up" the pool with 16 concurrent tasks.
// This starts the tasks on the thread pool and
// returns immediately without blocking.
for (int i = 0; i < 16; i++) {
    pool.Return(pool.Get());
}

Usage:

// Get a pooled task or create a new one. The task may
// have already completed, in which case Result will
// be available immediately. If the task is still
// in flight, accessing its Result will block.
Task<MyComponent> task = pool.Get();

try
{
    MyComponent component = task.Result; // Alternatively you can "await task"

    // Do something with component.
}
finally
{
    pool.Return(task);
}

This method is more complex than maintaining your component in a ThreadLocal or thread static field, but if you need to do something fancy like limiting the number of pooled instances, the pool abstraction can be quite useful.

EDIT

Basic "fixed set of X instances" pool implementation with a Get which blocks once the pool has been drained:

public sealed class ObjectPool<T>
{
    private readonly Queue<T> __objects;

    public ObjectPool(IEnumerable<T> items)
    {
        __objects = new Queue<T>(items);
    }

    public T Get()
    {
        lock (__objects)
        {
            while (__objects.Count == 0) {
                Monitor.Wait(__objects);
            }

            return __objects.Dequeue();
        }
    }

    public void Return(T obj)
    {
        lock (__objects)
        {
            __objects.Enqueue(obj);

            Monitor.Pulse(__objects);
        }
    }
}