Java Stream: aggregate in list all not-Null collections received in map()

Question

I have collection idList that contains string IDs. Function getCollection returns for single ID a collection of items (type MyType). Also, it may return null.

So for many IDs from idList I would get some nulls and some collections.

The goal is to collect all replies of getCollection for a set of IDs into final List.

I have imagined something like

 List<MyType> reply = idList.stream().map(id -> getCollection(id))
       .filter(p -> p != null).collect(Collectors.toList());

but it does not seem to be a valid expression. How to make it valid?

Also, what about performance of this implementation?

Answer 1

You need to use flatMap -

List<MyType> reply = idList.stream()
    .map(id -> getCollection(id))
    .filter(collection -> collection != null)
    .flatMap(Collection::stream)
    .collect(Collectors.toList());

If you are thinking about the differences between the map and the flatMap operation, then you can consult this excellent answer -

Both map and flatMap can be applied to a Stream and they both return a Stream. The difference is that the map operation produces one output value for each input value, whereas the flatMap operation produces an arbitrary number (zero or more) values for each input value.

This is reflected in the arguments to each operation.

The map operation takes a Function, which is called for each value in the input stream and produces one result value, which is sent to the output stream.

The flatMap operation takes a function that conceptually wants to consume one value and produce an arbitrary number of values. However, in Java, it's cumbersome for a method to return an arbitrary number of values, since methods can return only zero or one value. One could imagine an API where the mapper function for flatMap takes a value and returns an array or a List of values, which are then sent to the output. Given that this is the streams library, a particularly apt way to represent an arbitrary number of return values is for the mapper function itself to return a stream! The values from the stream returned by the mapper are drained from the stream and are passed to the output stream. The "clumps" of values returned by each call to the mapper function are not distinguished at all in the output stream, thus the output is said to have been "flattened."

Typical use is for the mapper function of flatMap to return Stream.empty() if it wants to send zero values, or something like Stream.of(a, b, c) if it wants to return several values. But of course any stream can be returned.