How to flatten all items from a nested Java Collection into a single List?

Question

Given a complex nested collection of objects such as:

Set<List<Map<String, List<Object>>>> complexNestedCollection;

Does a generic method exist to flatten this out and get a single List of all Objects contained within?

A few details:

1. The list shouldn't include collection objects themselves or map keys - only the values at the lowest level.
2. It should follow the same ordering where possible - so in the example, items in the list would be in order, whereas ordering of maps/sets would depend on the implementation.
3. It could optionally exclude duplicates
4. UPDATE: It should ideally detect/handle circular references at any level, e.g. a List<List<Object>> where the outer List contains itself as a member. (Credit to Adrian Jałoszewski for mentioning this in the comments below).

Note: The actual use case is to get all Strings from a List<List<String>>, which can be done easily enough with two loops but it made me wonder about the general case.

Answer 1

Assuming that you use Java 8, you could do that with the Stream API thanks to the method flatMap(Function<? super T,? extends Stream<? extends R>> mapper) as next:

// 1. Convert the Set as a Stream of List<Map<String, List<Object>>>
// 2. Extract the elements of the lists to get a Stream of Map<String, List<Object>>
// 3. Extract values of the maps to get a Stream of List<Object>
// 4. Extract the elements of the lists to get a Stream of Object
// 5. Get rid of duplicates
// 6. Collect the result as a List of Object
List<Object> result = complexNestedCollection.stream()
    .flatMap(List::stream)
    .flatMap(m -> m.values().stream())
    .flatMap(List::stream)
    .distinct()
    .collect(Collectors.toList());

<R> Stream<R> flatMap(Function<? super T,? extends Stream<? extends R>> mapper)

Returns a stream consisting of the results of replacing each element of this stream with the contents of a mapped stream produced by applying the provided mapping function to each element. Each mapped stream is closed after its contents have been placed into this stream. (If a mapped stream is null an empty stream is used, instead.)

---

For previous versions of Java, you can still use FluentIterable from Google Guava to replace the Stream and use transformAndConcat(Function<? super E,? extends Iterable<? extends T>> function) instead of flatMap to flatten your collection.

The previous code snippet would then be rewritten as next:

List<Object> result =
    new ArrayList<>(
        new LinkedHashSet<>(
            FluentIterable.from(complexNestedCollection)
                .transformAndConcat(
                    new Function<List<Map<String, List<Object>>>, Iterable<Map<String, List<Object>>>> () {
                        public Iterable<Map<String, List<Object>>> apply(final List<Map<String, List<Object>>> input) {
                            return input;
                        }
                    }
                ).transformAndConcat(
                    new Function<Map<String, List<Object>>, Iterable<List<Object>>> () {
                        public Iterable<List<Object>> apply(final Map<String, List<Object>> input) {
                            return input.values();
                        }
                    }
                ).transformAndConcat(
                    new Function<List<Object>, Iterable<Object>> () {
                        public Iterable<Object> apply(final List<Object> input) {
                            return input;
                        }
                    }
                ).toList()
        )
    );

Answer 2

I'm not sure if this exact implementation would work, as it's full of unchecked warnings and other dangerous stuff, but you should get the general idea.

public static Set<Object> recursiveExtract(Object stuff) {

    Set<Object> set = new HashSet<Object>();

    if(stuff instanceof Iterable) {
        for(Object o : (Iterable<?>)stuff) {
            set.addAll(recursiveExtract(o));
        }
    } else if(stuff instanceof Map) {
        for(Object o : ((Map<?, ? extends Object>) stuff).values()) {
            set.addAll(recursiveExtract(o));
        }
    } else {
        set.add(stuff);
    }

    return set;
}

You can also use List<Object> if you insist on List, but then you could get duplicate results, or LinkedHashSet<Object> if you care about the order.

---

Please instead of downvotes, give me suggestions for improvement. It's nicer.