Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Very slow foreach loop

Tags:

c#

.net

ado.net

I am working on an existing application. This application reads data from a huge file and then, after doing some calculations, it stores the data in another table.

But the loop doing this (see below) is taking a really long time. Since the file sometimes contains 1,000s of records, the entire process takes days.

Can I replace this foreach loop with something else? I tried using Parallel.ForEach and it did help. I am new to this, so will appreciate your help.

foreach (record someredord Somereport.r)
{
    try
    {
        using (var command = new SqlCommand("[procname]", sqlConn))
        {
            command.CommandTimeout = 0;
            command.CommandType = CommandType.StoredProcedure;
            command.Parameters.Add(…);

            IAsyncResult result = command.BeginExecuteReader();
            while (!result.IsCompleted)
            {
                System.Threading.Thread.Sleep(10);
            }
            command.EndExecuteReader(result);
        }
    }
    catch (Exception e)
    {
        …
    }
}

After reviewing the answers , I removed the Async and used edited the code as below. But this did not improve performance.

using (command = new SqlCommand("[sp]", sqlConn))
{
    command.CommandTimeout = 0;
    command.CommandType = CommandType.StoredProcedure;
    foreach (record someRecord in someReport.)
    {
        command.Parameters.Clear();
        command.Parameters.Add(....)
        command.Prepare();                            

        using (dr = command.ExecuteReader())
        {
            while (dr.Read())
            {
                if ()
                {

                }
                else if ()
                {

                }
            }
        }                             
    }                        
}
like image 231
user1110790 Avatar asked Aug 30 '12 17:08

user1110790


People also ask

Is foreach slower than for loop?

Foreach performance is approximately 6 times slower than FOR / FOREACH performance. The FOR loop without length caching works 3 times slower on lists, comparing to arrays. The FOR loop with length caching works 2 times slower on lists, comparing to arrays.

Is foreach slow PHP?

Kickstart HTML, CSS and PHP: Build a Responsive WebsiteThe 'foreach' is slow in comparison to the 'for' loop. The foreach copies the array over which the iteration needs to be performed. For improved performance, the concept of references needs to be used. In addition to this, 'foreach' is easy to use.

Is foreach slower than a for loop C#?

The forloop is faster than the foreach loop if the array must only be accessed once per iteration.


1 Answers

Instead of looping the sql connection so many times, ever consider extracting the whole set of data out from sql server and process the data via the dataset?

Edit: Decided to further explain what i meant.. You can do the following, pseudo code as follow

  1. Use a select * and get all information from the database and store them into a list of the class or dictionary.
  2. Do your foreach(record someRecord in someReport) and do the condition matching as usual.
like image 82
Guo Hong Lim Avatar answered Sep 27 '22 20:09

Guo Hong Lim