Java 8: List files from multiple paths

Question

How to search files from multiple paths in Java 8. These are not sub/sibling directories. For example, if I want to search json files in a path, I have:

try (Stream<Path> stream = Files.find(Paths.get(path), Integer.MAX_VALUE, (p, attrs) -> attrs.isRegularFile() && p.toString().endsWith(".json"))) {
  stream.map((p) -> p.name).forEach(System.out::println);
}

Is there a better way to search in multiple paths? Or do I have to run the same code for multiple paths?

Answer 1

Yes you can do it. Assuming you have paths as a List of String objects, you can do it like so,

List<String> paths = ...;

paths.stream().map(path -> {
    try (Stream<Path> stream = Files.list(Paths.get(path))) {
        return stream.filter(p -> !p.toFile().isDirectory()).filter(p -> p.toString().endsWith(".json"))
                .map(Path::toString).collect(Collectors.joining("\n"));
    } catch (IOException e) {
        // Log your ERROR here.
        e.printStackTrace();
    }
    return "";
}).forEach(System.out::println);

In case if you need to get rid of the new-line character, then it can be done like this too.

paths.stream().map(path -> {
    try (Stream<Path> stream = Files.walk(Paths.get(path))) {
        return stream.filter(p -> !p.toFile().isDirectory()).filter(p -> p.toString().endsWith(".json"))
                .map(Path::toString).collect(Collectors.toList());
    } catch (IOException e) {
        e.printStackTrace();
    }
    return Collections.emptyList();
}).flatMap(List::stream).forEach(System.out::println);

Here you get all the .json file names for each path into a List, and then flatten them into a flat stream of String objects before printing. Notice that the additional step involved in this approach which is flatMap.

Answer 2

That’s what flatMap is for.

If you have a collection of Path instances in path, you may use

paths.stream()
     .flatMap(path -> {
         try { return Files.find(path, Integer.MAX_VALUE,
            (p, attrs) -> attrs.isRegularFile() && p.toString().endsWith(".json")); }
         catch (IOException ex) { throw new UncheckedIOException(ex); }
     })
     .forEach(System.out::println);

It is a bit clumsy due to the fact that we have to deal with the checked IOException declared by find here. Rethrowing it as UncheckedIOException is the simplest choice as that’s what the stream returned by find will do anyway if an I/O problem occurs while processing the stream. Compare with the documentation of find

If an IOException is thrown when accessing the directory after returned from this method, it is wrapped in an UncheckedIOException which will be thrown from the method that caused the access to take place.

So doing the same within our function simplifies the caller’s error handling, as it just has to handle UncheckedIOExceptions.

There is no way to use try(…) here, but that’s exactly the reason why flatMap will remove this burden from us. As its documentation states:

Each mapped stream is closed after its contents have been placed into this stream.

So, once our function has returned the sub-stream, the Stream mplementation will do the right thing for us.

You may chain arbitrary stream operations in place of .forEach(System.out::println); to process all elements of the flattened stream like a single stream.

If your input collection contains instances of String instead of Path, you may simply prepend paths.stream().map(Paths::get) instead of paths.stream() to the flatMap operation.