Why doesn't Scala's Option[T] translate directly to a T in bytecode?

Question

One thing I don't understand about Scala, is why Null subclasses everything, despite not passing the substitution test. Presumably this is for Java compatibility, which is fine, I guess -- but then Scala pushes the Option[T] pattern.

This I don't understand. An Option[T] doesn't give you any extra guarantees (as every T is a defacto Option anyway). But it also makes a total of 4 states:

val a: Option[String] = null
val b: Option[String] = Some(null)
val c: Option[String] = None
val d: Option[String] = Some("A string")

This seems both inefficient (from a bytecode pov) and perhaps even worse than just pain Java. What my question is, why didn't Scala make Option[T] a special case that translates directly to a java bytecode's T. All interfacing with Java code (that uses references) would have to be via this Option[T] (which really, is exactly what it is). And there would be an annotation or something, for when a Scala method works with a T that can't be None.

This seems like the most obviously correct, most typesafe and most efficient.

Thanks

Answer 1

The main reason for Option is to support the Any type which can be an AnyVal or AnyRef. Scala requires that AnyVal cannot be null. This means that a type parameter T, can not simply be set to null. This makes it awkward to use generics in libraries like the scala collections library without something like Option.

val n: Option[Int] = Some(null) // will not compile

I believe you are right that there are many possible optimizations for eliminating Option that can eliminate the actual use of the class at the byte code level. There is an option to turn on aggressive optimizations for the Scala compiler which may do just that but it is still being heavily worked on.

Answer 2

Neil's answer is pretty good, but I'd like to add another factor. Option is not part of the language, it is just a library. How would you implement what you suggest?

Also, though I don't think this is that big of a deal, you can reflect the type of an Option, which would just not be the case if it were just stored as T.

Answer 3

There are three possible reasons:

1. Scala is a thin wrapper, that only adds new stuff and they don't want to break that.
2. They don't want to make Java libraries a pain in the ass to use, because all the arguments would then be Option[T] instead of T.
3. They don't want to have to use name-mangling or annotations to differentiate libraries written in Scala and written in Java (to mark a method as taking a "non-nullable reference").