How to iterate nested lists with lambda streams?

Question

I'm trying to refactor the following code to lambda expressions with `stream, especially the nested foreach loops:

public static Result match (Response rsp) {
    Exception lastex = null;

    for (FirstNode firstNode : rsp.getFirstNodes()) {
        for (SndNode sndNode : firstNode.getSndNodes()) {
            try {
                if (sndNode.isValid())
                return parse(sndNode); //return the first match, retry if fails with ParseException
            } catch (ParseException e) {
                lastex = e;
            }
        }
    }

    //throw the exception if all elements failed
    if (lastex != null) {
        throw lastex;
    }

    return null;
}

I'm starting with:

rsp.getFirstNodes().forEach().?? // how to iterate the nested 2ndNodes?

Answer 1

Look at flatMap:

flatMap(Function<? super T,? extends Stream<? extends R>> mapper)
Returns a stream consisting of the results of replacing each element of this stream with the contents of a mapped stream produced by applying the provided mapping function to each element.

Code sample assuming isValid() doesn't throw

Optional<SndNode> sndNode = rsp.getFirstNodes()
  .stream()
  .flatMap(firstNode -> firstNode.getSndNodes().stream())  //This is the key line for merging the nested streams
  .filter(sndNode -> sndNode.isValid())
  .findFirst();

if (sndNode.isPresent()) {
    try {
        parse(sndNode.get());
    } catch (ParseException e) {
        lastex = e;
    }
}

Answer 2

I am afraid that using streams and lambdas, your performance may suffer. Your current solution returns the first valid and parse-able node, however it is not possible to interrupt an operation on stream such as for-each (source).

Also, because you can have two different outputs (returned result or thrown exception), it won't be possible to do this with single line expression.

Here is what I came up with. It may give you some ideas:

public static Result match(Response rsp) throws Exception {
    Map<Boolean, List<Object>> collect = rsp.getFirstNodes().stream()
            .flatMap(firstNode -> firstNode.getSndNodes().stream()) // create stream of SndNodes
            .filter(SndNode::isValid) // filter so we only have valid nodes
            .map(node -> {
                // try to parse each node and return either the result or the exception
                try {
                    return parse(node);
                } catch (ParseException e) {
                    return e;
                }
            }) // at this point we have stream of objects which may be either Result or ParseException
            .collect(Collectors.partitioningBy(o -> o instanceof Result)); // split the stream into two lists - one containing Results, the other containing ParseExceptions

    if (!collect.get(true).isEmpty()) {
        return (Result) collect.get(true).get(0);
    }
    if (!collect.get(false).isEmpty()) {
        throw (Exception) collect.get(false).get(0); // throws first exception instead of last!
    }
    return null;
}

As mentioned at the beginning, there is possible performance issue as this will try to parse every valid node.

---

EDIT:

To avoid parsing all nodes, you could use reduce, but it is a bit more complex and ugly (and extra class is needed). This also shows all ParseExceptions instead of just last one.

private static class IntermediateResult {

    private final SndNode node;
    private final Result result;
    private final List<ParseException> exceptions;

    private IntermediateResult(SndNode node, Result result, List<ParseException> exceptions) {
        this.node = node;
        this.result = result;
        this.exceptions = exceptions;
    }

    private Result getResult() throws ParseException {
        if (result != null) {
            return result;
        }
        if (exceptions.isEmpty()) {
            return null;
        }
        // this will show all ParseExceptions instead of just last one
        ParseException exception = new ParseException(String.format("None of %s valid nodes could be parsed", exceptions.size()));
        exceptions.stream().forEach(exception::addSuppressed);
        throw exception;
    }

}

public static Result match(Response rsp) throws Exception {
    return Stream.concat(
                    Arrays.stream(new SndNode[] {null}), // adding null at the beginning of the stream to get an empty "aggregatedResult" at the beginning of the stream
                    rsp.getFirstNodes().stream()
                            .flatMap(firstNode -> firstNode.getSndNodes().stream())
                            .filter(SndNode::isValid)
            )
            .map(node -> new IntermediateResult(node, null, Collections.<ParseException>emptyList()))
            .reduce((aggregatedResult, next) -> {
                if (aggregatedResult.result != null) {
                    return aggregatedResult;
                }

                try {
                    return new IntermediateResult(null, parse(next.node), null);
                } catch (ParseException e) {
                    List<ParseException> exceptions = new ArrayList<>(aggregatedResult.exceptions);
                    exceptions.add(e);
                    return new IntermediateResult(null, null, Collections.unmodifiableList(exceptions));
                }
            })
            .get() // aggregatedResult after going through the whole stream, there will always be at least one because we added one at the beginning
            .getResult(); // return Result, null (if no valid nodes) or throw ParseException
}

---

EDIT2:

In general, it is also possible to use lazy evaluation when using terminal operators such as findFirst(). So with a minor change of requirements (i.e. returning null instead of throwing exception), it should be possible to do something like below. However, flatMap with findFirst doesn't use lazy evaluation (source), so this code tries to parse all nodes.

private static class ParsedNode {
    private final Result result;

    private ParsedNode(Result result) {
        this.result = result;
    }
}

public static Result match(Response rsp) throws Exception {
    return rsp.getFirstNodes().stream()
            .flatMap(firstNode -> firstNode.getSndNodes().stream())
            .filter(SndNode::isValid)
            .map(node -> {
                try {
                    // will parse all nodes because of flatMap
                    return new ParsedNode(parse(node));
                } catch (ParseException e ) {
                    return new ParsedNode(null);
                }
            })
            .filter(parsedNode -> parsedNode.result != null)
            .findFirst().orElse(new ParsedNode(null)).result;
}