How to optimize large size for loop

Question

I have a for loop with more than 20k iterations,for each iteration it is taking around two or three seconds and total around 20minutes. how i can optimize this for loop. I am using .net3.5 so parallel foreach is not possible. so i splited the 200000 nos into small chunks and implemented some threading now i am able reduce the time by 50%. is there any other way to optimize these kind of for loops.

My sample code is given below

    static double sum=0.0;
    public double AsyncTest()
    {
            List<Item> ItemsList = GetItem();//around 20k items
            int count = 0;
            bool flag = true;
            var newItemsList = ItemsList.Take(62).ToList();
            while (flag)
            {
                int j=0;
                WaitHandle[] waitHandles = new WaitHandle[62];
                foreach (Item item in newItemsList)
                {
                    var delegateInstance = new MyDelegate(MyMethod);
                    IAsyncResult asyncResult = delegateInstance.BeginInvoke(item.id, new AsyncCallback(MyAsyncResults), null);
                    waitHandles[j] = asyncResult.AsyncWaitHandle;
                    j++;
                }
                WaitHandle.WaitAll(waitHandles);
                count = count + 62;
                newItemsList = ItemsList.Skip(count).Take(62).ToList();  
            }
            return sum;
    }

    public double MyMethod(int id)
    {
        //Calculations
        return sum;
    }

    static public void MyAsyncResults(IAsyncResult iResult)
    {
        AsyncResult asyncResult = (AsyncResult) iResult;
        MyDelegate del = (MyDelegate) asyncResult.AsyncDelegate;
        double mySum = del.EndInvoke(iResult);
        sum = sum + mySum;
    }

Answer 1

It's possible to reduce number of loops by various techniques. However, this won't give you any noticeable improvement since the heavy computation is performed inside your loops. If you've already parallelized it to use all your CPU cores there is not much to be done. There is a certain amount of computation to be done and there is a certain computer power available. You can't squeeze from your machine more than it can provide.

You can try to:

1. Do a more efficient implementation of your algorithm if it's possible
2. Switch to faster environment/language, such as unmanaged C/C++.

Answer 2

1. Is there a rationale behind your batches size (62)?
2. Is "MyMethod" method IO bound or CPU bound?

What you do in each cycle is wait till all the batch completes and this wastes some cycles (you are actually waiting for all 62 calls to complete before taking the next batch). Why won't you change the approach a bit so that you still keep N operations running simultaneosly, but you fire a new operation as soon as one of the executind operations completes?