How to find and remove duplicate objects in a collection using LINQ?

Question

I have a simple class representing an object. It has 5 properties (a date, 2 decimals, an integer and a string). I have a collection class, derived from CollectionBase, which is a container class for holding multiple objects from my first class.

My question is, I want to remove duplicate objects (e.g. objects that have the same date, same decimals, same integers and same string). Is there a LINQ query I can write to find and remove duplicates? Or find them at the very least?

Answer 1

You can remove duplicates using the Distinct operator.

There are two overloads - one uses the default equality comparer for your type (which for a custom type will call the Equals() method on the type). The second allows you to supply your own equality comparer. They both return a new sequence representing your original set without duplicates. Neither overload actually modifies your initial collection - they both return a new sequence that excludes duplicates..

If you want to just find the duplicates, you can use GroupBy to do so:

var groupsWithDups = list.GroupBy( x => new { A = x.A, B = x.B, ... }, x => x ) 
                         .Where( g => g.Count() > 1 );

To remove duplicates from something like an IList<> you could do:

yourList.RemoveAll( yourList.Except( yourList.Distinct() ) );

Answer 2

If your simple class uses Equals in a manner that satisfies your requirements then you can use the Distinct method

var col = ...;
var noDupes = col.Distinct();

If not then you will need to provide an instance of IEqualityComparer<T> which compares values in the way you desire. For example (null problems ignored for brevity)

public class MyTypeComparer : IEqualityComparer<MyType> {
  public bool Equals(MyType left, MyType right) {
    return left.Name == right.Name;
  }
  public int GetHashCode(MyType type) {
    return 42;
  }
}

var noDupes = col.Distinct(new MyTypeComparer());

Note the use of a constant for GetHashCode is intentional. Without knowing intimate details about the semantics of MyType it is impossible to write an efficient and correct hashing function. In lieu of an efficient hashing function I used a constant which is correct irrespective of the semantics of the type.