Java 8 Stream List<Foo> to Map<Date, Map<String,Long>> with conditional groupingBy

Question

Following class:

public class Foo {
    private Date date;
    private String name;
    private Long number;
}

I now have a List<Foo> which I want to convert to Map<Date, Map<String,Long>> (Long should be a sum of numbers). What makes this hard is that I want exactly 26 entries in the inner map, where the 26th is called "Others" which sums up everything that has a number lower than the other 25.

I came up with following code:

data.stream().collect(Collectors.groupingBy(e -> e.getDate(), Collectors.groupingBy(e -> {
    if (/*get current size of inner map*/>= 25) {
        return e.getName();
    } else {
        return "Other";
    }

}, Collectors.summingLong(e -> e.getNumber()))));

As you can see, I have no idea how to check the number of elements which are already in the inner map. How can I get the current size of the inner map or is there another way to achieve what I want?

My Java 7 code:

Map<Date, Map<String, Long>> result = new LinkedHashMap<Date, Map<String, Long>>();
for (Foo fr : data) {
    if (result.get(fr.getDate()) == null) {
        result.put(fr.getDate(), new LinkedHashMap<String, Long>());
    }
    if (result.get(fr.getDate()) != null) {
        if (result.get(fr.getDate()).size() >= 25) {
            if (result.get(fr.getDate()).get("Other") == null) {
                result.get(fr.getDate()).put("Other", 0l);
            }
            if (result.get(fr.getDate()).get("Other") != null) {
                long numbers= result.get(fr.getDate()).get("Other");
                result.get(fr.getDate()).replace("Other", numbers+ fr.getNumbers());
            }
        } else {
            result.get(fr.getDate()).put(fr.getName(), fr.getNumbers());
        }
    }
}

Edit:

The map should help me to realize a table like this:

But I need to sum the "Others" first.

---

_{If you need any more infos feel free to ask}

Answer 1

I don’t think that this operation will benefit from using the Stream API. Still, you can improve the operation with Java 8 features:

Map<Date, Map<String, Long>> result = new LinkedHashMap<>();
for(Foo fr : data) {
    Map<String, Long> inner
      = result.computeIfAbsent(fr.getDate(), date -> new LinkedHashMap<>());
    inner.merge(inner.size()>=25?"Other":fr.getAirlineName(), fr.getNumbers(), Long::sum);
}

This code assumes that the airline names are already unique for each date. Otherwise, you would have to extend the code to

Map<Date, Map<String, Long>> result = new LinkedHashMap<>();
for(Foo fr : data) {
    Map<String, Long> inner
      = result.computeIfAbsent(fr.getDate(), date -> new LinkedHashMap<>());
    inner.merge(inner.size() >= 25 && !inner.containsKey(fr.getAirlineName())?
      "Other": fr.getAirlineName(), fr.getNumbers(), Long::sum);
}

to accumulate the values for the airline correctly.

---

For completeness, here is how to implement it as a stream operation.

Since the custom collector has some complexity, it’s worth writing it as reusable code:

public static <T,K,V> Collector<T,?,Map<K,V>> toMapWithLimit(
    Function<? super T, ? extends K> key, Function<? super T, ? extends V> value,
    int limit, K fallBack, BinaryOperator<V> merger) {

    return Collector.of(LinkedHashMap::new, (map, t) ->
            mergeWithLimit(map, key.apply(t), value.apply(t), limit, fallBack, merger),
            (map1,map2) -> {
                if(map1.isEmpty()) return map2;
                if(map1.size()+map2.size() < limit)
                    map2.forEach((k,v) -> map1.merge(k, v, merger));
                else
                    map2.forEach((k,v) ->
                        mergeWithLimit(map1, k, v, limit, fallBack, merger));
                return map1;
            });
}
private static <T,K,V> void mergeWithLimit(Map<K,V> map, K key, V value,
    int limit, K fallBack, BinaryOperator<V> merger) {
    map.merge(map.size() >= limit && !map.containsKey(key)? fallBack: key, value, merger);
}

This is like Collectors.toMap, but supporting a limit and a fallback key for additional entries. You may recognize the Map.merge call, similar to the loop solution as the crucial element.

Then, you may use the collector as

Map<Date, Map<String, Long>> result = data.stream().collect(
    Collectors.groupingBy(Foo::getDate, LinkedHashMap::new,
        toMapWithLimit(Foo::getAirlineName, Foo::getNumbers, 25, "Other", Long::sum)));

Answer 2

A bit too late :) But I come with this Java 8 solution without using the for loop or the custom collector. It is based on collectingAndThen which allows you to transform the result of collecting operation.

It allows me to divide the stream in finisher operation based on treshold.

However, I am not sure about the performance.

 int treshold = 25


 Map<Date, Map<String, Long>> result = data.stream().collect(groupingBy(Foo::getDate,
            collectingAndThen(Collectors.toList(), x -> {
                if (x.size() >= treshold) {
                    Map<String, Long> resultMap = new HashMap<>();
                    resultMap.putAll(x.subList(0, treshold).stream().collect(groupingBy(Foo::getName, Collectors.summingLong(Foo::getNumber))));
                    resultMap.putAll(x.subList(treshold, x.size()).stream().collect(groupingBy(y -> "Other", Collectors.summingLong(Foo::getNumber))));
                    return resultMap;
                } else {
                    return x.stream().collect(groupingBy(Foo::getName, Collectors.summingLong(Foo::getNumber)));
                }
            })));

Answer 3

First of all, let's simplify the original problem by adapting it to java 8 without using Streams.

Map<Date, Map<String, Long>> result = new LinkedHashMap();
for (Foo fr : data) {
    Map<String, Long> map = result.getOrDefault(fr.getDate(), new LinkedHashMap());
    if (map.size() >= 25) {
        Long value = map.getOrDefault("Other", 0L); // getOrDefault from 1.8
        map.put("Other", value + 1);
    } else {
        map.put(fr.getName(), fr.getNumber());
    }
    result.put(fr.getDate(), map);
}

And now using Stream

int limit = 25;
Map<Date, Map<String, Long>> collect = data.stream()
    .collect(Collectors.groupingBy(Foo::getDate))
    .entrySet().stream()
    .collect(Collectors.toMap(Map.Entry::getKey, v -> {
        Map<String, Long> c = v.getValue().stream()
                .limit(limit)
                .collect(Collectors.toMap(Foo::getName, Foo::getNumber));
        long remaining = v.getValue().size() - limit;
        if (remaining > 0) {
            c.put("Other", remaining);
        }
        return c;
    }));