Why is a type parameter stronger then a method parameter

Question

Why is

public <R, F extends Function<T, R>> Builder<T> withX(F getter, R returnValue) {...}

more strict then

public <R> Builder<T> with(Function<T, R> getter, R returnValue) {...}

This is a follow up on Why is lambda return type not checked at compile time. I found using the method withX() like

.withX(MyInterface::getLength, "I am not a Long")

produces the wanted compile time error:

The type of getLength() from the type BuilderExample.MyInterface is long, this is incompatible with the descriptor's return type: String

while using the method with() does not.

full example:

import java.util.function.Function;

public class SO58376589 {
  public static class Builder<T> {
    public <R, F extends Function<T, R>> Builder<T> withX(F getter, R returnValue) {
      return this;
    }

    public <R> Builder<T> with(Function<T, R> getter, R returnValue) {
      return this;
    }

  }

  static interface MyInterface {
    public Long getLength();
  }

  public static void main(String[] args) {
    Builder<MyInterface> b = new Builder<MyInterface>();
    Function<MyInterface, Long> getter = MyInterface::getLength;
    b.with(getter, 2L);
    b.with(MyInterface::getLength, 2L);
    b.withX(getter, 2L);
    b.withX(MyInterface::getLength, 2L);
    b.with(getter, "No NUMBER"); // error
    b.with(MyInterface::getLength, "No NUMBER"); // NO ERROR !!
    b.withX(getter, "No NUMBER"); // error
    b.withX(MyInterface::getLength, "No NUMBER"); // error !!!
  }
}

javac SO58376589.java

SO58376589.java:32: error: method with in class Builder<T> cannot be applied to given types;
    b.with(getter, "No NUMBER"); // error
     ^
  required: Function<MyInterface,R>,R
  found: Function<MyInterface,Long>,String
  reason: inference variable R has incompatible bounds
    equality constraints: Long
    lower bounds: String
  where R,T are type-variables:
    R extends Object declared in method <R>with(Function<T,R>,R)
    T extends Object declared in class Builder
SO58376589.java:34: error: method withX in class Builder<T> cannot be applied to given types;
    b.withX(getter, "No NUMBER"); // error
     ^
  required: F,R
  found: Function<MyInterface,Long>,String
  reason: inference variable R has incompatible bounds
    equality constraints: Long
    lower bounds: String
  where F,R,T are type-variables:
    F extends Function<MyInterface,R> declared in method <R,F>withX(F,R)
    R extends Object declared in method <R,F>withX(F,R)
    T extends Object declared in class Builder
SO58376589.java:35: error: incompatible types: cannot infer type-variable(s) R,F
    b.withX(MyInterface::getLength, "No NUMBER"); // error
           ^
    (argument mismatch; bad return type in method reference
      Long cannot be converted to String)
  where R,F,T are type-variables:
    R extends Object declared in method <R,F>withX(F,R)
    F extends Function<T,R> declared in method <R,F>withX(F,R)
    T extends Object declared in class Builder
3 errors

Extended Example

The following example shows the different behaviour of method and type parameter boiled down to a Supplier. In addition it shows the difference to a Consumer behaviour for a type parameter. And it shows it does not make a difference wether it is a Consumer or Supplier for a method parameter.

import java.util.function.Consumer;
import java.util.function.Supplier;
interface TypeInference {

  Number getNumber();

  void setNumber(Number n);

  @FunctionalInterface
  interface Method<R> {
    TypeInference be(R r);
  }

  //Supplier:
  <R> R letBe(Supplier<R> supplier, R value);
  <R, F extends Supplier<R>> R letBeX(F supplier, R value);
  <R> Method<R> let(Supplier<R> supplier);  // return (x) -> this;

  //Consumer:
  <R> R lettBe(Consumer<R> supplier, R value);
  <R, F extends Consumer<R>> R lettBeX(F supplier, R value);
  <R> Method<R> lett(Consumer<R> consumer);


  public static void main(TypeInference t) {
    t.letBe(t::getNumber, (Number) 2); // Compiles :-)
    t.lettBe(t::setNumber, (Number) 2); // Compiles :-)
    t.letBe(t::getNumber, 2); // Compiles :-)
    t.lettBe(t::setNumber, 2); // Compiles :-)
    t.letBe(t::getNumber, "NaN"); // !!!! Compiles :-(
    t.lettBe(t::setNumber, "NaN"); // Does not compile :-)

    t.letBeX(t::getNumber, (Number) 2); // Compiles :-)
    t.lettBeX(t::setNumber, (Number) 2); // Compiles :-)
    t.letBeX(t::getNumber, 2); // !!! Does not compile  :-(
    t.lettBeX(t::setNumber, 2); // Compiles :-)
    t.letBeX(t::getNumber, "NaN"); // Does not compile :-)
    t.lettBeX(t::setNumber, "NaN"); // Does not compile :-)

    t.let(t::getNumber).be(2); // Compiles :-)
    t.lett(t::setNumber).be(2); // Compiles :-)
    t.let(t::getNumber).be("NaN"); // Does not compile :-)
    t.lett(t::setNumber).be("NaN"); // Does not compile :-)
  }
}

Answer 1

This is a really interesting question. The answer, I'm afraid, is complicated.

tl;dr

Working out the difference involves some quite in-depth reading of Java's type inference specification, but basically boils down to this:

• All other things equal, the compiler infers the most specific type it can.
• However, if it can find a substitution for a type parameter that satisfies all the requirements, then compilation will succeed, however vague the substitution turns out to be.
• For with there is a (admittedly vague) substitution that satisfies all the requirements on R: Serializable
• For withX, the introduction of the additional type parameter F forces the compiler to resolve R first, without considering the constraint F extends Function<T,R>. R resolves to the (much more specific) String which then means that inference of F fails.

This last bullet point is the most important, but also the most hand-wavy. I can't think of a better concise way of phrasing it, so if you want more details, I suggest you read the full explanation below.

Is this intended behaviour?

I'm gonna go out on a limb here, and say no.

I'm not suggesting there's a bug in the spec, more that (in the case of withX) the language designers have put their hands up and said "there are some situations where type inference gets too hard, so we'll just fail". Even though the compiler's behaviour with respect to withX seems to be what you want, I would consider that to be an incidental side-effect of the current spec, rather than a positively intended design decision.

This matters, because it informs the question Should I rely on this behaviour in my application design? I would argue that you shouldn't, because you can't guarantee that future versions of the language will continue to behave this way.

While it's true that language designers try very hard not to break existing applications when they update their spec/design/compiler, the problem is that the behaviour you want to rely on is one where the compiler currently fails (i.e. not an existing application). Langauge updates turn non-compiling code into compiling code all the time. For example, the following code could be guaranteed not to compile in Java 7, but would compile in Java 8:

static Runnable x = () -> System.out.println();

Your use-case is no different.

Another reason I'd be cautious about using your withX method is the F parameter itself. Generally, a generic type parameter on a method (that doesn't appear in the return type) exists to bind the types of multiple parts of the signature together. It's saying:

I don't care what T is, but want to be sure that wherever I use T it's the same type.

Logically, then, we would expect each type parameter to appear at least twice in a method signature, otherwise "it's not doing anything". F in your withX only appears once in the signature, which suggests to me a use of a type parameter not inline with the intent of this feature of the language.

An alternative implementation

One way to implement this in a slightly more "intended behaviour" way would be to split your with method up into a chain of 2:

public class Builder<T> {

    public final class With<R> {
        private final Function<T,R> method;

        private With(Function<T,R> method) {
            this.method = method;
        }

        public Builder<T> of(R value) {
            // TODO: Body of your old 'with' method goes here
            return Builder.this;
        }
    }

    public <R> With<R> with(Function<T,R> method) {
        return new With<>(method);
    }

}

This can then be used as follows:

b.with(MyInterface::getLong).of(1L); // Compiles
b.with(MyInterface::getLong).of("Not a long"); // Compiler error

This doesn't include an extraneous type parameter like your withX does. By breaking down the method into two signatures, it also better expresses the intent of what you're trying to do, from a type-safety point of view:

• The first method sets up a class (With) that defines the type based on the method reference.
• The scond method (of) constrains the type of the value to be compatible with what you previously set up.

The only way a future version of the language would be able to compile this is if the implemented full duck-typing, which seems unlikely.

One final note to make this whole thing irrelevant: I think Mockito (and in particular its stubbing functionality) might basically already do what you're trying to achieve with your "type safe generic builder". Maybe you could just use that instead?

The full(ish) explanation

I'm going to work through the type inference procedure for both with and withX. This is quite long, so take it slowly. Despite being long, I've still left quite a lot of details out. You may wish to refer to the spec for more details (follow the links) to convince yourself that I'm right (I may well have made a mistake).

Also, to simplify things a little, I'm going to use a more minimal code sample. The main difference is that it swaps out Function for Supplier, so there are less types and parameters in play. Here's a full snippet that reproduces the behaviour you described:

public class TypeInference {

    static long getLong() { return 1L; }

    static <R> void with(Supplier<R> supplier, R value) {}
    static <R, F extends Supplier<R>> void withX(F supplier, R value) {}

    public static void main(String[] args) {
        with(TypeInference::getLong, "Not a long");       // Compiles
        withX(TypeInference::getLong, "Also not a long"); // Does not compile
    }

}

Let's work through the type applicability inference and type inference procedure for each method invocation in turn:

with

We have:

with(TypeInference::getLong, "Not a long");

The initial bound set, B₀, is:

• R <: Object

All parameter expressions are pertinent to applicability.

Hence, the initial constraint set for applicability inference, C, is:

• TypeInference::getLong is compatible with Supplier<R>
• "Not a long" is compatible with R

This reduces to bound set B₂ of:

• R <: Object (from B₀)
• Long <: R (from the first constraint)
• String <: R (from the second constraint)

Since this does not contain the bound 'false', and (I assume) resolution of R succeeds (giving Serializable), then the invocation is applicable.

So, we move on to invocation type inference.

The new constraint set, C, with associated input and output variables, is:

• TypeInference::getLong is compatible with Supplier<R>
  • Input variables: none
  • Output variables: R

This contains no interdependencies between input and output variables, so can be reduced in a single step, and the final bound set, B₄, is the same as B₂. Hence, resolution succeeds as before, and the compiler breathes a sigh of relief!

withX

We have:

withX(TypeInference::getLong, "Also not a long");

The initial bound set, B₀, is:

• R <: Object
• F <: Supplier<R>

Only the second parameter expression is pertinent to applicability. The first one (TypeInference::getLong) is not, because it meets the following condition:

If m is a generic method and the method invocation does not provide explicit type arguments, an explicitly typed lambda expression or an exact method reference expression for which the corresponding target type (as derived from the signature of m) is a type parameter of m.

Hence, the initial constraint set for applicability inference, C, is:

• "Also not a long" is compatible with R

This reduces to bound set B₂ of:

• R <: Object (from B₀)
• F <: Supplier<R> (from B₀)
• String <: R (from the constraint)

Again, since this does not contain the bound 'false', and resolution of R succeeds (giving String), then the invocation is applicable.

Invocation type inference once more...

This time, the new constraint set, C, with associated input and output variables, is:

• TypeInference::getLong is compatible with F
  • Input variables: F
  • Output variables: none

Again, we have no interdependencies between input and output variables. However this time, there is an input variable (F), so we must resolve this before attempting reduction. So, we start with our bound set B₂.

1. We determine a subset V as follows:

   Given a set of inference variables to resolve, let V be the union of this set and all variables upon which the resolution of at least one variable in this set depends.

   By the second bound in B₂, the resolution of F depends on R, so V := {F, R}.

2. We pick a subset of V according to the rule:

   let { α1, ..., αn } be a non-empty subset of uninstantiated variables in V such that i) for all i (1 ≤ i ≤ n), if αi depends on the resolution of a variable β, then either β has an instantiation or there is some j such that β = αj; and ii) there exists no non-empty proper subset of { α1, ..., αn } with this property.

   The only subset of V that satisfies this property is {R}.

3. Using the third bound (String <: R) we instantiate R = String and incorporate this into our bound set. R is now resolved, and the second bound effectively becomes F <: Supplier<String>.

4. Using the (revised) second bound, we instantiate F = Supplier<String>. F is now resolved.

Now that F is resolved, we can proceed with reduction, using the new constraint:

1. TypeInference::getLong is compatible with Supplier<String>
2. ...reduces to Long is compatible with String
3. ...which reduces to false

... and we get a compiler error!

---

Additional notes on the 'Extended Example'

The Extended Example in the question looks at a few interesting cases that aren't directly covered by the workings above:

• Where the value type is a subtype of the method return type (Integer <: Number)
• Where the functional interface is contravariant in the inferred type (i.e. Consumer rather than Supplier)

In particular, 3 of the given invocations stand out as potentially suggesting 'different' compiler behaviour to that described in the explanations:

t.lettBe(t::setNumber, "NaN"); // Does not compile :-)

t.letBeX(t::getNumber, 2); // !!! Does not compile  :-(
t.lettBeX(t::setNumber, 2); // Compiles :-)

The second of these 3 will go through exactly the same inference process as withX above (just replace Long with Number and String with Integer). This illustrates yet another reason why you shouldn't rely on this failed type inference behaviour for your class design, as the failure to compile here is likely not a desirable behaviour.

For the other 2 (and indeed any of the other invocations involving a Consumer you wish to work through), the behaviour should be apparent if you work through the type inference procedure laid out for one of the methods above (i.e. with for the first, withX for the third). There's just one small change you need to take note of:

• The constraint on the first parameter (t::setNumber is compatible with Consumer<R>) will reduce to R <: Number instead of Number <: R as it does for Supplier<R>. This is described in the linked documentation on reduction.

I leave it as an excercise for the reader to carfully work through one of the above procedures, armed with this piece of additional knowledge, to demonstrate to themselves exactly why a particular invocation does or doesn't compile.