Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove duplicates from list based on multiple fields or columns

I have a list of type MyClass

public class MyClass
{
   public string prop1 {} 
   public int prop2 {} 
   public string prop3 {} 
   public int prop4 {} 
   public string prop5 {} 
   public string prop6 {} 
   ....
}

This list will have duplicates. I want to find and remove items from this list where prop1, prop2 and prop3 are duplicates. It doesnt matter if the other properties are duplicates

This is what I have tried that is not working.

List<MyClass> noDups = myClassList.GroupBy(d => new {d.prop1,d.prop2,d.prop3} ).Where(g => g.Count() > 1).Select(g=> g.Key);

I dont want to use any third party tools for this. Only pure linq.

like image 691
user20358 Avatar asked Feb 25 '15 09:02

user20358


People also ask

How do I remove duplicate rows based on multiple columns in SQL?

In SQL, some rows contain duplicate entries in multiple columns(>1). For deleting such rows, we need to use the DELETE keyword along with self-joining the table with itself.

How do you drop duplicate rows in pandas based on multiple columns?

By using pandas. DataFrame. drop_duplicates() method you can remove duplicate rows from DataFrame. Using this method you can drop duplicate rows on selected multiple columns or all columns.

How do you remove duplicates based on criteria Excel?

In Excel, there are several ways to filter for unique values—or remove duplicate values: To filter for unique values, click Data > Sort & Filter > Advanced. To remove duplicate values, click Data > Data Tools > Remove Duplicates.


1 Answers

This will return one item for each "type" (like a Distinct) (so if you have A, A, B, C it will return A, B, C)

List<MyClass> noDups = myClassList.GroupBy(d => new {d.prop1,d.prop2,d.prop3} )
                                  .Select(d => d.First())
                                  .ToList();

If you want only the elements that don't have a duplicate (so if you have A, A, B, C it will return B, C):

List<MyClass> noDups = myClassList.GroupBy(d => new {d.prop1,d.prop2,d.prop3} )
                                  .Where(d => d.Count() == 1)
                                  .Select(d => d.First())
                                  .ToList();
like image 194
xanatos Avatar answered Oct 23 '22 19:10

xanatos