Unexpected generic behavior with TypeToken nesting generic types

Question

I have TypeToken class used to represent some generic type like this:
TypeToken<List<String>> listOfStrings = new TypeToken<List<String>> {}
And this works fine, TypeToken is just class TypeToken<T> {} with simple method to get that type.

Now I wanted to create simple methods for common type like List for more dynamic usage:
TypeToken<List<? extends Number>> numbers = list(extendsType(Number.class))
using:

public static <T> TypeToken<? extends T> extendsType(Class<T> type) {return null;}
public static <T> TypeToken<List<T>> list(TypeToken<T> type) {return null;}

(return nulls as I'm only asking about compiler not logic)

But for some reason this does not work how I would expect: (as code that I expected to be valid does not compile, and code that I expected to be invalid does compile)

class TypeToken<X> {
    static <T> TypeToken<? extends T> extendsType(Class<T> type) {return null;}
    static <T> TypeToken<List<T>> list(TypeToken<T> type) {return null;}
    static void wat() {
        TypeToken<List<? extends Number>> a = new TypeToken<List<? extends Number>>() {}; // valid
        TypeToken<List<? extends Number>> b = list(extendsType(Number.class)); // invalid, why?
        TypeToken<? extends List<? extends Number>> c = list(extendsType(Number.class)); // valid, why?
    }
}

What I'm doing wrong here? And what is causing generics to behave like this?

I'm using JDK 11, but I also tested this on JDK 8

Compiler error:

error: incompatible types: no instance(s) of type variable(s) T#1,CAP#1,T#2 exist so that TypeToken<List<T#1>> conforms to TypeToken<List<? extends Number>>
            TypeToken<List<? extends Number>> b = list(extendsType(Number.class)); // invalid, why?
                                                      ^
  where T#1,T#2 are type-variables:
    T#1 extends Object declared in method <T#1>list(TypeToken<T#1>)
    T#2 extends Object declared in method <T#2>extendsType(Class<T#2>)
  where CAP#1 is a fresh type-variable:
    CAP#1 extends T#2 from capture of ? extends T#2

Answer 1

I think that at the core this question that has already been asked in a similar form several times. But I'm not sure, because this is one of the constellations where it is particularly hard to wrap one's head around the concept.

Maybe one can imagine it like that:

• The extendsType method here returns a TypeToken<? extends Number>

• In the call to the list method, the ? is captured in the type parameter T. This capture can (roughly speaking) be imagined as a new type variable - something like X extends Number, and the method returns a TypeToken<List<X>>

• Now the TypeToken<List<X>> is not assignable to TypeToken<List<? extends Number>>, as of the usual constraints. Maybe this table, with <=== indicating assignability and <=/= indicating non-assignability, will help:

                      Number                     <===                Integer
  TypeToken<          Number>                    <=/=      TypeToken<Integer>
  TypeToken<? extends Number>                    <===      TypeToken<Integer>
                      List<? extends Number>     <===                List<X>
  TypeToken<          List<? extends Number>>    <=/=      TypeToken<List<X>>
  TypeToken<? extends List<? extends Number>>    <===      TypeToken<List<X>>
  

So in the end, the answer to the question which super-subtype relationships exist among instantiations of generic types in the famous Generics FAQ by Angelika Langer is probably once more the most relevant here:

Super-subtype relationships among instantiations of generic types are determined by two orthogonal aspects.

On the one hand, there is the inheritance relationship between a supertype and a subtype.

...

On the other hand, there is a relationship based on the type arguments. The prerequisite is that at least one of the involved type arguments is a wildcard. For example, Collection<? extends Number> is a supertype of Collection<Long>, because the type Long is a member of the type family that the wildcard " ? extends Number " denotes.

---

^{I think that the idea of the capture being a "new type X" sounds convincing, even though there is some handwaving involved: This happens somewhat "implicitly" in the resolution process, and not visible in the code, but I found it helpful, to some extent, when I wrote a library for types where all these questions came up...}

---

Updated to elaborate this further, referring to the comment:

I mentioned that the types are not assignable "as of the usual constraints". Now one could argue about where these "constraints" come from. But they basically always have the same reason: The types are not assignable, because if they were assignable, the program would not be type safe. (Meaning that it would be possible to provoke a ClassCastException one way or the other).

Explaining where the type safety is lost here (and why a ClassCastException could be caused) involves some contortions. I'll try to show it here, based on the example that was referred to in the comment. Scroll down to tl;dr for a an example that has the same structure, but is much simpler.

The example from the comment was this:

import java.util.*;
import java.io.*;

class Ideone
{
    public static class LinkTypeToken<T> extends TypeToken<List<T>> {}
    public static class TypeToken<T> {}
    public static void main (String[] args) throws java.lang.Exception  {            
        LinkTypeToken<Number> listA = null;
        TypeToken<List<Number>> listB = listA;

        LinkTypeToken<? extends Number> listC = null;
        TypeToken<? extends List<? extends Number>> listD = listC;

        // error: incompatible types: LinkTypeToken<CAP#1> cannot be 
        // converted to TypeToken<List<? extends Number>>
        // TypeToken<List<? extends Number>> listE = listC;
        //                                           ^
        // where CAP#1 is a fresh type-variable:
        //   CAP#1 extends Number from capture of ? extends Number        
        TypeToken<List<? extends Number>> listE = listC;
    }
}

The line that causes the compilation error here does so because if it was possible to assign these types, then one could do the following:

import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;

class IdeoneWhy
{
    public static class ArrayListTypeToken<T> extends TypeToken<ArrayList<T>>
    {
        ArrayList<T> element = null;
        void setElement(ArrayList<T> element)
        {
            this.element = element;
        }
    }

    public static abstract class TypeToken<T>
    {
        abstract void setElement(T element);
    }

    public static void main(String[] args)
    {
        ArrayListTypeToken<? extends Number> listC = new ArrayListTypeToken<Integer>();
        TypeToken<? extends List<? extends Number>> listD = listC;

        // This is not possible:
        //TypeToken<List<? extends Number>> listE = listC;

        // But let's enforce it with a brutal cast:
        TypeToken<List<? extends Number>> listE = 
            (TypeToken<List<? extends Number>>)(Object)listC;

        // This throws a ClassCastException
        listE.setElement(new LinkedList<Integer>());
    }
}

So the fact that a TypeToken<List<? extends Number>> is not assignable from a TypeToken<? extends List<? extends Number>> indeed only serves the purpose of preventing a ClassCastException.

tl;dr :

The simpler variant is this:

import java.util.ArrayList;
import java.util.List;

public class WhySimpler
{
    public static void main(String[] args)
    {
        List<Float> floats = new ArrayList<Float>();

        // This is not possible
        //List<Number> numbers = floats;

        // Let's enforce it with a brutal cast:
        List<Number> numbers = (List<Number>)(Object)floats;

        Integer integer = 123;

        // This is possible, because Integer is a Number: 
        numbers.add(integer);

        // Now, we ended up placing an Integer into a list that
        // may only contain Float values.
        // So this will cause a ClassCastException:
        Float f = floats.get(0);

    }
}

Conversely, when the type is declared as List<? extends Number>, then the assignment is possible, because it is not possible to sneak an invalid type into such a list:

List<Float> floats = new ArrayList<Float>();
List<? extends Number> numbers = floats;
numbers.add(someInteger); // This is not possible

Long story short: It's all about type safety.