Scala type bounds and Java generic interop

Question

I'm trying to wrap the rxjava's timeout method as to make it available for scala.

Similar to many other methods I tried this:

def timeout[U >: T](timeout: Duration, other: Observable[U]): Observable[U] = {
  val otherJava: rx.Observable[_ <: U] = other.asJavaObservable
  val thisJava:  rx.Observable[_ <: U] = this.asJavaObservable
  toScalaObservable[U](thisJava.timeout(timeout.length, timeout.unit, otherJava))
}

But I'm getting the following error:

Observable.scala:1631: error: overloaded method value timeout with alternatives:
($1: Long,x$2: java.util.concurrent.TimeUnit,x$3: rx.Scheduler)rx.Observable[_$85] <and>
($1: Long,x$2: java.util.concurrent.TimeUnit,x$3: rx.Observable[_ <: _$85])rx.Observable[_$85]
cannot be applied to (Long, scala.concurrent.duration.TimeUnit, rx.Observable[_$84])
  toScalaObservable[U](thisJava.timeout(timeout.length, timeout.unit, otherJava))

The original java method:

public Observable<T> timeout(long timeout, TimeUnit timeUnit, Observable<? extends T> other) {
   return create(OperationTimeout.timeout(this, timeout, timeUnit, other));
}

I'm not very familiar with neither Java nor Scala (and all the type bounds) but as far as I understand: Both otherJava as well as thisJava is of type rx.Observable[U], so why don't they line up?

Answer 1

Hum, your are stepping right on the variance problems of Java generics used in Scala. Let's go step by step.

---

Let's look at your implementation:

// does not compile (with your original error)
def timeout[U >: T](timeout: Duration, other: Observable[U]): Observable[U] = {
  val otherJava: rx.Observable[_ <: U] = other.asJavaObservable
  val thisJava:  rx.Observable[_ <: U] = this.asJavaObservable
  toScalaObservable[U](thisJava.timeout(timeout.length, timeout.unit, otherJava))
}

To understand why this won't work, let's call A the unnamed type in the declaration of thisJava (the A <: U such that thisJava is a rx.Observable[A]). The timeout method of thisJava: rx.Observable[A] expects a parameter of type rx.Observable[_ <: A], and you give it one of type rx.Observable[_ <: U]: the compiler has no way to know how those two types are related. They might not be related at all!

On the other hand, if A were U, then thisJava would be a rx.Observable[U], and its timeout method would expect a rx.Observable[_ <: U], which just happen to be the type of otherJava. Let's try:

// still does not compile, sadly
def timeout[U >: T](timeout: Duration, other: Observable[U]): Observable[U] = {
  val otherJava: rx.Observable[_ <: U] = other.asJavaObservable
  val thisJava:  rx.Observable[U] = this.asJavaObservable // variance error
  toScalaObservable[U](thisJava.timeout(timeout.length, timeout.unit, otherJava))
}

In a perfect world, the above would work. However, the java rx.Observable isn't defined as covariant since there is no definition-site variance annotations in java. So Scala believes it is invariant.

Hence as far as Scala is concerned, a rx.Observable[_ <: U] is not a rx.Observable[U], and sadly this.asJavaObservable returns a rx.Observable[_ <: U].

---

But we know [*] that rx.Observable<T> should be covariant, so we can just blindly cast away:

// this compiles and *should* work
def timeout[U >: T](timeout: Duration, other: Observable[U]): Observable[U] = {
  val otherJava: rx.Observable[_ <: U] = other.asJavaObservable
  val thisJava = this.asJavaObservable.asInstanceOf[rx.Observable[U]]
  toScalaObservable[U](thisJava.timeout(timeout.length, timeout.unit, otherJava))
}

The moral of this story is that mixing Scala's variance and Java's variance will always cost you a few cast here and there, that must be carefully thought about.

Also, making asJavaObservable return a rx.Observable[T] instead of _ <: T would make all this easier, but maybe there are good reasons why it isn't the case...

[*] more like "but I suspect"

Answer 2

[this should go into the comments of @gourlaysama's answer, but I don't have enough reputation to comment]

@Aralo The statement "MyType[_ <: T] is the same as MyType[T]" only holds if the compiler knows that MyType is covariant. This is the case with List, because it's defined as List[+A], but it's not the case with rx.Observable, because it's a Java type, so its type parameters cannot have variance annotations, so the compiler cannot know that it's intended to be covariant.

@gourlaysama making asJavaObservable return a rx.Observable[T] instead of _ <: T is not the solution, because the type rx.lang.scala.Observable[T] means "an Observable of T or of something which is a subtype of T", and this description corresponds exactly to the type rx.Observable[_ <: T] (which is the same as rx.Observable<? extends T>).

The reason why we have to cast in Scala is that the signature of timeout in Java is "wrong": Strictly speaking, it makes the Java Observable invariant, because T appears in a contravariant position. The "correct" way would be to use another type parameter U, whose lower bound is T, as in Scala, but Java doesn't support lower bounds, so the best solution is to keep the "wrong" solution. This problem occurs also with reduce (see this comment), onErrorReturn, and a few other operators.

In general, all these operators which should have a lower bounded type parameter (but don't) can only be used on an Observable<T>, but not on an Observable<? extends T> (a quite serious inconvenience for Java users), and thus, they require a cast in the Scala adaptor.