Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Optimize LINQ to Objects query


I have around 200K records in a list and I'm looping through them and forming another collection. This works fine on my local 64 bit Win 7 but when I move it to a Windows Server 2008 R2, it takes a lot of time. There is difference of about an hour almost!

I tried looking at Compiled Queries and am still figuring it out.
For various reasons, we cant do a database join and retrieve the child values

Here is the code:

//listOfDetails is another collection
List<SomeDetails> myDetails = null;
foreach (CustomerDetails myItem in customerDetails)
{

    var myList = from ss in listOfDetails
                 where ss.CustomerNumber == myItem.CustomerNum
                 && ss.ID == myItem.ID
                 select ss;
     myDetails = (List<SomeDetails>)(myList.ToList());
     myItem.SomeDetails = myDetails;
}
like image 242
hangar18 Avatar asked Dec 12 '22 22:12

hangar18


1 Answers

I would do this differently:

var lookup = listOfDetails.ToLookup(x => new { x.CustomerNumber, x.ID });
foreach(var item in customerDetails)
{
    var key = new { CustomerNumber = item.CustomerNum, item.ID };
    item.SomeDetails = lookup[key].ToList();
}

The big benefit of this code is that it only has to loop through the listOfDetails once to build the lookup - which is nothing more than a hash map. After that we just get the values using the key, which is very fast as that is what hash maps are built for.

like image 81
Daniel Hilgarth Avatar answered Dec 14 '22 10:12

Daniel Hilgarth