Reified generic parameter inside coroutine is not working

Question

I am creating http json client. I am using Volley in combination with coroutines. I wanted to create generic http client so I can use it everywhere.

I have created generic extension method to parse JSON string into object.

inline fun <reified T>String.jsonToObject(exclusionStrategy: ExclusionStrategy? = null) : T {
val builder = GsonBuilder()

if(exclusionStrategy != null){
    builder.setExclusionStrategies(exclusionStrategy)
}

return builder.create().fromJson(this, object: TypeToken<T>() {}.type)

}

Problem is that when I call this method I don't get expected result. First call gives proper result. Object is initialized. But second call, where I use generic parameter which is passed to method, ends with exception "LinkedTreeMap can not be cast into Token".

    protected inline fun <reified T>sendRequestAsync(endpoint: String, data: Any?, method: Int, token: Token?): Deferred<T> {
    return ioScope.async {
        suspendCoroutine<T> { continuation ->
            val jsonObjectRequest = HttpClient.createJsonObjectRequest(
                endpoint,
                data?.toJsonString(),
                method,
                Response.Listener {
                    //this call is successful and object is initialized
                    val parsedObject : HttpResponse<Token> = it.toString().jsonToObject()

                    //this call is not successful and object is not initialized properly
                    val brokenObject : HttpResponse<T> = it.toString().jsonToObject()
                    continuation.resume(brokenObject.response)
                },
                Response.ErrorListener {
                    continuation.resumeWithException(parseException(it))
                },
                token)
            HttpClient.getInstance(context).addToRequestQueue(jsonObjectRequest)
        }
    }
}

Call of generic method.

fun loginAsync(loginData: LoginData): Deferred<Token> {
    return sendRequestAsync("/tokens/", loginData, Request.Method.POST, null)
}

This is how httpresponse data class looks.

data class HttpResponse<T> (
val response: T
)

I saw a workaround here using Type::class.java but I don't like this approach and I would like to use reified and inline keywords. How does the reified keyword in Kotlin work?

UPDATE This is exception which I am getting.

java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap cannot be cast to com.xbionicsphere.x_card.entities.Token

POSSIBLE WORKAROUND I found possible workaround. If I create method which will parse Token into from response and use this method in executeRequestAsync everything starts working but I don't like this solution since I have to add additional parameter for each request.

New loginAsync

fun loginAsync(loginData: LoginData): Deferred<Token> {
    val convertToResponse : (JSONObject) -> HttpResponse<Token> = {
        it.toString().jsonToObject()
    }

    return executeRequestAsync("/tokens/", loginData, Request.Method.POST, null, convertToResponse)
}

New executeRequestAsync

    protected inline fun <reified T>executeRequestAsync(endpoint: String, data: Any?, method: Int, token: Token?, crossinline responseProvider: (JSONObject) -> HttpResponse<T>): Deferred<T> {
    return ioScope.async {
        suspendCoroutine<T> { continuation ->
            val jsonObjectRequest =
                HttpClient.createJsonObjectRequest(
                    endpoint,
                    data?.toJsonString(),
                    method,
                    Response.Listener {
                        val response: HttpResponse<T> = responseProvider(it)
                        continuation.resume(response.response)
                    },
                    Response.ErrorListener {
                        continuation.resumeWithException(parseException(it))
                    },
                    token
                )
            HttpClient.getInstance(
                context
            ).addToRequestQueue(jsonObjectRequest)
        }
    }
}

UPDATE I probably have found working solution. executeRequestAsync needs final type definition provided through generic parameters so I enhanced declaration of method. Now method declaration looks like this:

    protected inline fun <reified HttpResponseOfType, Type>executeRequestAsync(endpoint: String, data: Any?, method: Int, token: Token?) : Deferred<Type> where HttpResponseOfType : HttpResponse<Type> {
    val scopedContext = context

    return ioScope.async {
        suspendCoroutine<Type> { continuation ->
            val jsonObjectRequest =
                HttpClient.createJsonObjectRequest(
                    endpoint,
                    data?.toJsonString(),
                    method,
                    Response.Listener {
                        val response: HttpResponseOfType = it.toString().jsonToObject()
                        continuation.resume(response.response)
                    },
                    Response.ErrorListener {
                        continuation.resumeWithException(parseException(it))
                    },
                    token
                )
            HttpClient.getInstance(
                scopedContext
            ).addToRequestQueue(jsonObjectRequest)
        }
    }
}

Thanks this complicated function declaration I can execute request with this call:

fun loginAsync(loginData: LoginData): Deferred<Token> {
    return executeRequestAsync("/tokens/", loginData, Request.Method.POST, null)
}

Answer 1

In order to understand why the second call behaves kind of strangely and why, as proposed by Leo Aso, removing the keywords inline and reified (which requires an inlinable function) also breaks the first call, you have to understand type erasure and how reified enables type reification first.

Note: The following code is written in Java since I'm more familiar with Java's than with Kotlin's syntax. Furthermore, this makes type erasure easier to explain.

The type parameter of a generic function isn't available at runtime; Generics are a "compile-time trick" only. This applies to both, Java and Kotlin (since Kotlin is able to run on the JVM). The process in which the generic type information is removed is called type erasure and happens during compilation. So how do generic functions work at runtime? Consider the following function which returns the most valuable element of an arbitrary collection.

<T> T findHighest(Comparator<T> comparator, Collection<? extends T> collection) {
    T highest = null;
    for (T element : collection) {
        if (highest == null || comparator.compare(element, highest) > 0)
            highest = element;
    }

    return highest;
}

As this function can be invoked with many different kinds of collections etc, the value of the type variable T might vary over time. To ensure compatiblity with all of them, the function gets refactored during type erasure. After type erasure has completed, the function will look somehow similar to this:

Object findHighest(Comparator comparator, Collection collection) {
    Object highest = null;
    for (Object element : collection) {
        if (highest == null || comparator.compare(element, highest) > 0)
            highest = element;
    }

    return highest;
}

During type erasure, type variables are replaced with their bound. In this case, the bound type is Object. Parameterized don't keep their generic type information generally.

However, if you'd compile the erased code, some problems will occur. Consider the following code (unerased) which calls the erased one:

Comparator<CharSequence> comp = ...
List<String> list = ...
String max = findHighest(comp, list);

As #findHighest(Comparator, Collection) now returns Object, the assignment in line 3 would be illegal. The compiler therefore inserts a cast there during type erasure.

...
String max = (String) findHighest(comp, list);

As the compiler always knows which cast it has to insert, type erasue doesn't cause any problems in most cases. However, it comes with a few restrictions: instanceof doesn't work, catch (T exception) is illegal (whereas throws T is allowed since the calling function knows what kind of exception it has to expect), etc. The restriction you had to fight with is the lack of reifiable (= full type information available at runtime) generic types (there are a few exceptions, but they do not matter in this context).

---

But wait, Kotlin has support for reified types, right? That's true, but as I've mentioned earlier, this is only true for inlinable functions. But why is that?

When a function which signature contains the keyword inline is invoked, the invoking code is replaced with the code of this function. As the "copied" code no longer has to be compatible with all kinds of types, it can be optimized for the context it is used in.

One possible optimization is to replace the type variables in the "copied code" (there's a lot more happening under the hood) before type erasure is done. The type information is therefore preserved and also available at runtime; it is indistinguishable from any other non-generic code.

---

Although both of your functions, #jsonToObject(ExclusionStrategy?) and #sendRequestAsync(String, Any?, Int, Token?), are marked as inlinable and have reifiable type parameters, there's still something you've missed: T is, at least in your call to #toJsonObject(ExclusionStrategy?), NOT reifiable.

One reason for this is your call to #suspendCoroutine(...). To understand why this is a problem, we have to look at its declaration first:

suspend inline fun <T> suspendCoroutine(
    crossinline block: (Continuation<T>) -> Unit
): T

The crossinline-keyword is problematic as it stops the compiler from inlining the code that is declared inside block. The lambda you pass to #suspendCoroutine will therefore be transfered into an anonymous inner class. Technically, this happens under the hood at runtime.

The generic type information is therefore NOT available anymore, at least not at runtime. At the point where you invoke #jsonToObject(...), the type variable T is erased to Object. The TypeToken Gson generates therefore looks like this:

TypeToken<HttpResponse<Object>>

Update: This is, as I've found after some further research, NOT true. crossinline doesn't stop the compiler from inlining lambdas, it just forbids them to influence the function's control flow. I probably mixed it up with the keyword noinline, which, as the name implies, actually forbids inlining.

However, I'm pretty sure about the following part. However, I still have to find out why Gson is unable to determine and/or to deserialize the type correctly. I'll update this post as soon as I know more.

---

That brings us to the final part which tries to explain the weird exception you received. For that, we have to take a look at Gsons' internals.

Internally, Gson has two main types that are responsible for reflective serialization and deserialization: TypeAdapterFactory and TypeAdapter<T>.

A TypeAdapter<T> only adapts (= provides the (de-)serialization logic for) one specific type. This means that Integer, Double, List<String> and List<Float> are all handled by different TypeAdapter<T>s.

TypeAdapterFactorys are responsible for, as their names already imply, providing matching TypeAdapter<T>s. The differentiation between TypeAdapter<T>s and TypeAdapterFactorys is extremely useful as one factory might create all adapters for e.g. a collection type like List as they all work in a similar way.

In order to determine what kind of adapter you need, Gson expects you to pass a TypeToken<T> when calling a (de-)serialization function which should process a generic type. TypeToken<T> uses a "trick" to access the type information passed to its type parameter.

As soon as you call Gson#fromJson(this, object: TypeToken<T>() {}.type), Gson iterates through all available TypeAdapterFactorys until it finds one that can provide an appropriate TypeAdapter<T>. Gson comes with a variety of TypeAdapterFactorys, including factories for primitive data types, wrapper types, basic collection types, date and many more. Besides that, Gson provides two special factories:

• ReflectiveTypeAdapterFactory As the name already implies, this factory tries to access the object's data reflectively. In order to adapt the type of each field appropriately, it requests one matching TypeAdapter per field. This is the factory that will be selected for (de-)serialzing HttpRequest.
• ObjectTypeAdapter.Factory This factory only returns an ObjectTypeAdapter. The snippet below shows what it does on object deserialization (respectively the field in your HttpRequest object):

@Override public Object read(JsonReader in) throws IOException {
    JsonToken token = in.peek();
    switch (token) {
    ...
    case BEGIN_OBJECT:
      Map<String, Object> map = new LinkedTreeMap<String, Object>(); // <-----
      in.beginObject();
      while (in.hasNext()) {
        map.put(in.nextName(), read(in));
      }
      in.endObject();
      return map;   // <-----
    ...
  }

That's why you get a ClassCastException with a com.google.gson.internal.LinkedTreeMap.