C# ThreadPool application performance degrading over time

Question

I've a class, say "MyComputation" which does a lot of computation in one long constructor. It takes typically about 20ms to run when executed on its own (with no disk i/o or network operations). 100 or so instances of this class are created by a parent class, say "ComputeParent", which queues them up in a ThreadPool as work items:

ThreadPool.QueueUserWorkItem(myComputationCall, my_computation_data);

"myComputationCall" looks like this:

    public static void myComputationCall(Object my_computation_data)
    {
        try
        {
            MyDataObject data = (MyDataObject)my_computation_data;

            var computation_run = new MyComputation(data.parameter1, data.parameter2);

            data.result = computation_run.result;
        }
        finally
        {
            if (Interlocked.Decrement(ref num_work_items_remaining) == 0)
                done_event.Set();
        }
    }

done_event is a static ManualResetEvent:

    private static ManualResetEvent done_event;

    ...

    done_event = new ManualResetEvent(false);

I run ComputeParent about 500 or so times, for various input parameters. So I have a lot of nested classes. The problem is that the time it takes to execute ComputeParent gradually increases. There will be a certain amount of variation between how long it takes to run each particular ComputeParent, but the amount of time increases quite steadily (geometrically, each successive iteration take longer by a longer amount).

The memory consumption of the program does not noticably increase over time though it is quite high (~300MB). Its running on a computer with 8 logical cores, and the processor use seems to be very bursty. I'm not sure what else might be relevant to the problem.

I'd prefer not to have to run ComputeParent through batch files, though the issue does not appear to arise when this is done.

Answer 1

If number of available threads in the ThreadPool becomes 0, and you continue to add new work items then newly added work items will "wait". This means that your ComputeParent will wait for its instances of "myComputationCall". Starting more and more ComputeParent will cause that average execution time of them will go up.