Running maximum threads: Automatic performance adjustment

Question

I'm developing an app which one scans thousands copies of a struct; ~1 GB RAM. Speed is important.

     ParallelScan(_from, _to);  //In a new thread

I manually adjust the threads count:

     if (myStructs.Count == 0) { threads = 0; }
     else if (myStructs.Count < 1 * Number.Thousand) { threads = 1; }
     else if (myStructs.Count < 3 * Number.Thousand) { threads = 2; }
     else if (myStructs.Count < 5 * Number.Thousand) { threads = 4; }
     else if (myStructs.Count < 10 * Number.Thousand) { threads = 8; }
     else if (myStructs.Count < 20 * Number.Thousand) { threads = 12; }
     else if (myStructs.Count < 30 * Number.Thousand) { threads = 20; }
     else if (myStructs.Count < 50 * Number.Thousand) { threads = 30; }
     else threads = 40;

I just wrote it from scratch and I need to modify it for another CPU, etc. I think I could write a smarter code which one dynamically starts a new thread if CPU is available at the moment:

• If CPU is not %100 start N thread
• Measure CPU or thread process time & modify/estimate N
• Loop until scan all struct array

Is there anyone think that "I did something similar" or "I have a better idea" ?

UPDATE: The solution

    Parallel.For(0, myStructs.Count - 1, (x) =>
    {
         ParallelScan(x, x); // Will be ParallelScan(x);

    });

I did trim tons of code. Thanks people!

UPDATE 2: Results

Scan time for 10K templates

• 1 Thread: 500 ms
• 10 Threads: 300 ms
• 40 Threads: 600 ms
• Tasks: 100 ms

Answer 1

The standard answer: Use Tasks (TPL) , not Threads. Tasks require Fx4.

Your ParallelScan could just use Parallel.Foreach( ... ) or PLINQ (.AsParallel()).

The TPL framework includes a scheduler, and ForEach() uses a partitioner, to adapt to CPU cores and load. Your problem is most likely solved with the standard components but you can write custom-schedulers and -partitioners.

Answer 2

Actually, you won't get much benefit from spanning 50 threads, if you CPU only has two cores (even if each of them supports hyperthreading). If will actually run slower due to context switching which will occur every once in a while.

That means you should go for the Task Parallel Library (.NET 4), which takes care that all available cores are used efficiently.

Apart from that, improving the asymptotic duration of your search algorithm might prove more valuable for large quantities of data, regardless of the Moore's law.

[Edit]

If you are unable/unwilling to use .NET 4 TPL, you can start by getting the information about the current number of logical processors in the system (use Environment.ProcessorCount or check this answer for detailed info). Based on that number, you can partition your data and span a fixed number of threads. That is much simpler that checking the CPU utilization, and should prevent creating unnecessary threads which are starved anyway.