Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using LINQ vs SQL for Filtering Collection

Tags:

c#

asp.net

linq

I have a very general question regarding the use of LINQ vs SQL to filter a collection. Lets say you are running a fairly complex filter on a database table. It's running, say 10,000 times and the filters could be different every time. Performance wise, are you better off loading the entire database table collection into memory and executing the filters with LINQ, or should you let the database handle the filtering with SQL (since that's what is was built to do). Any thoughts?

EDIT: I should have been more clear. Lets assume we're talking about a table with 1000 records with 20 columns (containing int/string/date data). Currently in my app I am running one query every 1/2 hour to pull in all of the data into a collection (saving that collection in the application cache) and filtering that cached collection throughout my app. I'm wondering if that is worse than doing tons of round trips to the database server (it's Oracle fwiw).

like image 365
Rob Lauer Avatar asked Oct 08 '22 07:10

Rob Lauer


1 Answers

After the update:

It's running, say 10,000 times and

I'm going to assume a table with 1000 records

It seems reasonable to assume the 1k records will fit easily in memory.

And then running 10k filters will be much cheaper in memory (LINQ).
Using SQL would mean loading 10M records, a lot of I/O.

like image 180
Henk Holterman Avatar answered Oct 13 '22 11:10

Henk Holterman