C# .net For() Step?

Question

I have a function, that processes a list of 6100 list items. The code used to work when the list was just 300 items. But instantly crashes with 6100. Is there a way I can loop through these 6100 items say 30 at a time and execute a new thread per item?

    for (var i = 0; i < ListProxies.Items.Count; i++)
    {
        var s = ListProxies.Items[i] as string;
        var thread = new ParameterizedThreadStart(ProxyTest.IsAlive);
        var doIt = new Thread(thread) { Name = "CheckProxy# " + i };
        doIt.Start(s);
    }

Any help would be greatly appreciated.

Answer 1

Do you really need to spawn a new thread for each work item? Unless there is a genuine need for this (if so, please tell us why), I would strongly recommend you use the Managed Thread Pool instead. This will give you the concurrency benefits you require, but without the resource requirements (as well as the creation, destruction and massive context-switching costs) of running thousands of threads. If you are on .NET 4.0, you might also want to consider using the Task Parallel Library.

For example:

for (var i = 0; i < ListProxies.Items.Count; i++)
{
   var s = ListProxies.Items[i] as string;
   ThreadPool.QueueUserWorkItem(ProxyTest.IsAlive, s);       
}

On another note, I would seriously consider renaming the IsAlive method (which looks like a boolean property or method) since:

1. It clearly has a void IsAlive(object) signature.
2. It has observable side-effects (from your comment that it "increment a progress bar and add a 'working' proxy to a new list").

Answer 2

There is a limit on the number of threads you can spawn. 6100 threads does seem quite a bit excesive.

I agree win Ani, you should look into a ThreadPool or even a Producer / Consumer process depending on what you are trying to accomplish.

There are quite a few processes for handling multi threaded applications but without knowing what you are doing in the start there really is no way to recommend any approach other than a ThreadPool or Producer / Consumer process (Queues with SyncEvents).

At any rate you really should try to keep the number of threads to a minimum otherwise you run the risk of thread locks, spin locks, wait locks, dead locks, race conditions, who knows what, etc...

If you want good information on threading with C# check out the book Concurrent Programming on Windows By Joe Duffy it is really helpful.