Lambda in Stream.map/filter not called

Question

I'm trying to find separate the duplicates and non-duplicates in a List by adding them to a Set and List while using Stream.filter and Stream.map

List<String> strings = Arrays.asList("foo", "bar", "foo", "baz", "foo", "bar");

Set<String> distinct = new HashSet<>();
List<String> extras = new ArrayList<>();

strings
  .stream()
  .filter(x -> !distinct.add(x))
  .map(extra -> extras.add(extra));

At the end of this, I expect distinct to be [foo, bar, baz] and extras to be [foo, foo, bar], since there are 2 extra instances of foo and 1 of bar. However, they are both empty after I run this.

The lambdas given to the stream are never being called, which I verified by trying to print inside map:

.map(extra -> {
  System.out.println(extra);
  return extras.add(extra);
})

This doesn't work when I try to use put with Map either. What am I doing wrong?

---

Note: There may be other questions similar to this, but I'm looking for a kind of canonical answer for why this sort of stuff doesn't work with Java 8's Streams. If you can make this a more general question (even if it means completely changing it), I'd appreciate it.

Answer 1

Both Stream#filter and Stream#map are intermediate operations, which means that they are evaluated lazily. According to the documentation:

Intermediate operations return a new stream. They are always lazy; executing an intermediate operation such as filter() does not actually perform any filtering, but instead creates a new stream that, when traversed, contains the elements of the initial stream that match the given predicate. Traversal of the pipeline source does not begin until the terminal operation of the pipeline is executed.

In any case, you should be using the appropriate methods to avoid errors like this; forEach should be used instead of map here as Stream#map is used to convert the stream to the result of calling the mapping function on each element, while Stream#forEach is used to iterate over it.

Demo: https://ideone.com/ZQhLJC

strings
  .stream()
  .filter(x -> !distinct.add(x))
  .forEach(extras::add);

Another possible workaround is to perform a terminal operation like .collect to force the filter and map to be applied.

strings
  .stream()
  .filter(x -> !distinct.add(x))
  .map(extra -> extras.add(extra)).collect(Collectors.toList());

If you are going to use .collect, you might as well use the collected list as extras to avoid wasting time and space.

List<String> extras = strings
  .stream()
  .filter(x -> !distinct.add(x)).collect(Collectors.toList());

Answer 2

Your code does not work, because the stream is not consumed. You have provided only the intermediate operations, but until you call a terminating operation like forEach, reduce or collect, nothing you have defined in your stream will be invoked.

You should rather use peek to print the elements going through the stream and collect to get all the elements in the list:

List<String> extras = strings
    .stream()
    .filter(x -> !distinct.add(x))
    .peek(System.out::println)
    .collect(Collectors.toList());

Using forEach to fill the empty collection created before is a code smell and has nothing to do with functional programming.

Answer 3

In order to the filter to be applied, you need to call a terminal operation like collect(). In that case you can assign the items which pass the filter directly to the extras list instead of use map function.

Try something like this:

List<String> strings = Arrays.asList("foo", "bar", "foo", "baz", "foo", "bar");

Set<String> distinct = new HashSet<>();

List<String> extras = strings
                     .stream()
                     .filter(x -> !distinct.add(x))
                     .collect(Collectors.toList());