Creating complex objects using Collectors.groupingBy

Question

In oracle's reduction tutorial one can use Stream.collect to calculate the average age in a stream:

Averager averageCollect = roster.stream()
    .filter(p -> p.getGender() == Person.Sex.MALE)
    .map(Person::getAge)
    .collect(Averager::new, Averager::accept, Averager::combine);

But what if wanted to create a Map<Person.Sex, Averager> using lambda + groupingBy, instead of a simple average, as seen in the end of tutorial:

Map<Person.Sex, Integer> totalAgeByGender =
    roster
        .stream()
        .collect(
            Collectors.groupingBy(
                Person::getGender,                      
                Collectors.reducing(
                    0,
                    Person::getAge,
                    Integer::sum)));

Answer 1

Yes, this is a bit subtle. To change the value in the map, you have to change the downstream collector of the groupingBy call. In this case you have to apply a nested downstream collector.

The stream starts off with Persons and we want Averagers as the values of the map. To get from Person to Averager, we first need to map each Person to their age (an int) and then feed the ints to an Averager.

We start off by doing the grouping on gender, so the Persons corresponding to each gender need to be dealt with. The next step is to map the Persons to their ages using a mapping collector as the downstream collector of groupingBy.

Now that you have the ages, you want to create Averager instances for each group. The Averager class from the tutorial is already has the collector methods -- it supports the supplier, accumulator, and combiner functions which are suitable for passing to the Stream.collect call in the earlier example. Instead of Stream.collect, though, we want to use the Averager methods to form a nested downstream collector for the mapping collector we just established. Given these methods, a convenient way to create a Collector is using Collector.of.

You might try something like this:

Map<Person.Sex, Averager> map =
    roster.stream()
          .collect(groupingBy(Person::getGender,
              mapping(Person::getAge,
                  Collector.of(Averager::new, Averager::accept, Averager::combine))));

But wait! This doesn't work! You get a fairly nasty compilation failure that looks something like this:

error: no suitable method found for of(Averager::new,Averager::accept,Averager::combine)
                      Collector.of(Averager::new, Averager::accept, Averager::combine))));
method Collector.<T#1,R#1>of(Supplier<R#1>,BiConsumer<R#1,T#1>,BinaryOperator<R#1>,Characteristics...) is not applicable
  (cannot infer type-variable(s) T#1,R#1
    (argument mismatch; bad return type in method reference
      void cannot be converted to R#1))
method Collector.<T#2,A,R#2>of(Supplier<A>,BiConsumer<A,T#2>,BinaryOperator<A>,Function<A,R#2>,Characteristics...) is not applicable
  (cannot infer type-variable(s) T#2,A,R#2
    (argument mismatch; bad return type in method reference
      void cannot be converted to A))
where T#1,R#1,T#2,A,R#2 are type-variables:
T#1 extends Object declared in method <T#1,R#1>of(Supplier<R#1>,BiConsumer<R#1,T#1>,BinaryOperator<R#1>,Characteristics...)
R#1 extends Object declared in method <T#1,R#1>of(Supplier<R#1>,BiConsumer<R#1,T#1>,BinaryOperator<R#1>,Characteristics...)
T#2 extends Object declared in method <T#2,A,R#2>of(Supplier<A>,BiConsumer<A,T#2>,BinaryOperator<A>,Function<A,R#2>,Characteristics...)
A extends Object declared in method <T#2,A,R#2>of(Supplier<A>,BiConsumer<A,T#2>,BinaryOperator<A>,Function<A,R#2>,Characteristics...)
R#2 extends Object declared in method <T#2,A,R#2>of(Supplier<A>,BiConsumer<A,T#2>,BinaryOperator<A>,Function<A,R#2>,Characteristics...)

Ack! Actually, once you get past the intimidation factor of a 20-line error message, calm down, and read what it's trying to say, it's actually quite explicit about what the compiler is trying to do and how it's failing. You also have to look very carefully at the API.

The tutorial defines the three Averager methods for use in the Stream.collect method, which has the following signature (generics omitted for brevity):

collect(Supplier supplier, BiConsumer accumulator, BiConsumer combiner)

Note that the combiner method is a BiConsumer. However, the Collector.of method is defined as follows:

of(Supplier supplier, BiConsumer accumulator, BinaryOperator combiner, Collector.Characteristics... characteristics)

(The characteristics argument is varargs, and it doesn't concern us, so we can just omit it.)

The thing to notice here is that the combiner for Collector.of is a BinaryOperator instead of a BiConsumer. The BinaryOperator version does exactly the same thing as the BiConsumer version, but in addition, it returns the combined result. To fix this, we simply change the combine() method to return an Averager instead of void, and we add a return this statement:

public Averager combine(Averager other) {
    total += other.total;
    count += other.count;
    return this;
}

Note that this version of the combine() method is still suitable for passing as the third argument to Stream.collect. A BinaryOperator is compatible where a BiConsumer is required; the return value is simply ignored.

Once you've made this change to Averager.combine, this code (same as above) should work:

Map<Person.Sex, Averager> map =
    roster.stream()
          .collect(groupingBy(Person::getGender,
              mapping(Person::getAge,
                  Collector.of(Averager::new, Averager::accept, Averager::combine))));

Answer 2

The tutorial is about to explain the full details of how reduction works. However, for the simple cases you don’t need to implement everything by hand:

Map<Person.Sex, Double> map = roster.stream().collect(
    Collectors.groupingBy(
        Person::getGender,
        Collectors.averagingInt(Person::getAge)));

will give you a map from gender to average age.

But if you want to understand how to define your own reduction operation by specifying identity, mapper, and operation, look at Stuart Marks’s answer.