Java 8 remove duplicate strings irrespective of case from a list

Question

How can we remove duplicate elements from a list of String without considering the case for each word, for example consider below code snippet

    String str = "Kobe Is is The the best player In in Basketball basketball game .";
    List<String> list = Arrays.asList(str.split("\\s"));
    list.stream().distinct().forEach(s -> System.out.print(s+" "));

This still gives the same output as below, which is obvious

Kobe Is is The the best player In in Basketball basketball game .

I need the result as follows

Kobe Is The best player In Basketball game .

Answer 1

Taking your question literally, to “remove duplicate strings irrespective of case from a list”, you may use

// just for constructing a sample list
String str = "Kobe Is is The the best player In in Basketball basketball game .";
List<String> list = new ArrayList<>(Arrays.asList(str.split("\\s")));

// the actual operation
TreeSet<String> seen = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
list.removeIf(s -> !seen.add(s));

// just for debugging
System.out.println(String.join(" ", list));

Answer 2

In case you only need to get rid of consecutive duplicates, you can use a regular expression. The regex below checks for duplicated words, ignoring case.

String input = "Kobe Is is The the best player In in Basketball basketball game .";
String output = input.replaceAll("(?i)\\b(\\w+)\\s+\\1\\b", "$1");

System.out.println(output);

Which outputs:

Kobe Is The best player In Basketball game .

Answer 3

Here's a fun solution to get the expected result with the use of streams.

String result = Pattern.compile("\\s")
                .splitAsStream(str)
                .collect(Collectors.collectingAndThen(Collectors.toMap(String::toLowerCase,
                        Function.identity(),
                        (l, r) -> l,
                        LinkedHashMap::new),
                        m -> String.join(" ", m.values())));

prints:

Kobe Is The best player In Basketball game .

Answer 4

if it's not a problem for you losing while print all the capital letters, you can do in this way

    list.stream()
            .map(String::toLowerCase)
            .distinct()
            .forEach(System.out::print)

Output:

kobe is the best player in basketball game .