What is the best way to remove duplicates in an Array in Java?

Question

I have an Array of Objects that need the duplicates removed/filtered. I was going to just override equals & hachCode on the Object elements, and then stick them in a Set... but I figured I should at least poll stackoverflow to see if there was another way, perhaps some clever method of some other API?

Answer 1

I would agree with your approach to override hashCode() and equals() and use something that implements Set.

Doing so also makes it absolutely clear to any other developers that the non-duplicate characteristic is required.

Another reason - you get to choose an implementation that meets your needs best now:

• HashSet
• TreeSet
• LinkedHashSet

and you don't have to change your code to change the implementation in the future.

Answer 2

I found this in the web

Here are two methods that allow you to remove duplicates in an ArrayList. removeDuplicate does not maintain the order where as removeDuplicateWithOrder maintains the order with some performance overhead.

1. The removeDuplicate Method:

   /** List order not maintained **/
   public static void removeDuplicate(ArrayList arlList)
   {
    HashSet h = new HashSet(arlList);
    arlList.clear();
    arlList.addAll(h);
   }
   

2. The removeDuplicateWithOrder Method:

   /** List order maintained **/
   public static void removeDuplicateWithOrder(ArrayList arlList)
   {
      Set set = new HashSet();
      List newList = new ArrayList();
      for (Iterator iter = arlList.iterator(); iter.hasNext();) {
         Object element = iter.next();
         if (set.add(element))
            newList.add(element);
      }
      arlList.clear();
      arlList.addAll(newList);
   }
   

Answer 3

Overriding equals and hashCode and creating a set was my first thought too. It's good practice to have some overridden version of these methods anyway in your inheritance hierarchy.

I think that if you use a LinkedHashSet you'll even preserve order of unique elements...