Scala Script: Explain this class cast error to ScalaClassLoader on Windows

Question

Consider following scala script:

import scala.reflect.internal.util.ScalaClassLoader

object Test {
  def main(args: Array[String]) {
    val classloaderForScalaLibrary = classOf[ScalaClassLoader.URLClassLoader].getClassLoader
    println(classloaderForScalaLibrary)
    val classloaderForTestClass = this.getClass.getClassLoader
    println(classloaderForTestClass)
    this.getClass.getClassLoader.asInstanceOf[ScalaClassLoader.URLClassLoader]
  }
}

The output is:

scala.reflect.internal.util.ScalaClassLoader$URLClassLoader@71c8becc
scala.reflect.internal.util.ScalaClassLoader$URLClassLoader@71c8becc
java.lang.ClassCastException: scala.reflect.internal.util.ScalaClassLoader$URLClassLoader cannot be cast to scala.reflect.internal.util.ScalaClassLoader$URLClassLoader
        at Main$.main(Test.scala:8)
        at Main.main(Test.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at scala.reflect.internal.util.ScalaClassLoader.$anonfun$run$2(ScalaClassLoader.scala:98)
        at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:32)
...

Why can't I cast ScalaClassLoader$URLClassLoader to ScalaClassLoader$URLClassLoader?

Edit:

On running:

scala -J-verbose:class Test.scala | grep ScalaClassLoader

The output is:

[Loaded scala.reflect.internal.util.ScalaClassLoader$URLClassLoader from file:/C:/Development/Software/scala-2.12.2/lib/scala-reflect.jar]
...
...
[Loaded scala.reflect.internal.util.ScalaClassLoader$URLClassLoader from file:/C:/DEVELO~1/Software/SCALA-~1.2/lib/scala-reflect.jar]

So there is definitely some shady class loading going on. Now trying to investigate why this is so

Answer 1

If you extend your code a bit more as following:

import scala.reflect.internal.util.ScalaClassLoader

object test {

  def main(args: Array[String]) {
    val cl1 = this.getClass.getClassLoader
    println(cl1)
    val c1 = cl1.getClass 
    println(cl1.getClass)
    println(cl1.getClass.getClassLoader)

    println("-------")

    var c2 = classOf[ScalaClassLoader.URLClassLoader]
    println(c2)
    println(c2.getClassLoader)
    println("-------")
    println(c1 == c2)

  }
}

you'll get following output:

scala.reflect.internal.util.ScalaClassLoader$URLClassLoader@5cee5251
class scala.reflect.internal.util.ScalaClassLoader$URLClassLoader
sun.misc.Launcher$AppClassLoader@4554617c
-------
class scala.reflect.internal.util.ScalaClassLoader$URLClassLoader
scala.reflect.internal.util.ScalaClassLoader$URLClassLoader@5cee5251
-------
false

Notice matching hashes @5cee5251. It means that first Scala interpreter loads ScalaClassLoader$URLClassLoader using root Java class loader and then uses that class loader to load all the classes in your script and when you ask for ScalaClassLoader$URLClassLoader in your code it is loaded with another (already loaded) instance of ScalaClassLoader$URLClassLoader. In this way your script is isolated from the "runtime environment" that executes it.

You may find some details at ScalaClassLoader.asContext method, that you can see in your stack trace, that uses Thread.setContextClassLoader to set itself as the main classloader for the thread that executes your script.

Update (Why it works on Mac but doesn't work on Windows)

The major difference between scala shell script for *nix and scala.bat for Windows is that by default on *nix platforms standard Scala libraries are added to Boot Classpath (see usebootcp in the script) while on Windows they are added to the "Usual Classpath". This is important because it defines which class loader will load scala.reflect.internal.util.ScalaClassLoader that is used by scala.tools.nsc.MainGenericRunner: will it be the root class loader (which is represented as null if you call getClassLoader) or Application Class Loader (i.e. an instance of sun.misc.Launcher$AppClassLoader). This is important because CommonRunner.run creates an instance of ScalaClassLoader using just urls without parent

def run(urls: Seq[URL], objectName: String, arguments: Seq[String]) {
  (ScalaClassLoader fromURLs urls).run(objectName, arguments)
} 

This means that the parent class loader for the "main" ScalaClassLoader will be boot class loader rather than sun.misc.Launcher$AppClassLoader and thus when you ask this "main" ScalaClassLoader for class scala.reflect.internal.util.ScalaClassLoader it can't find it among classes loaded by its class loaders chain and thus has to load it again. This is the reason why you have two different instances of ScalaClassLoader class in your script.

There are two obvious workarounds (and both are not so good):

• Change CommonRunner.run in the Scala source to actually pass current context class loader as the parent to the new ScalaClassLoader (might be not that easy ☺)
• Change scala.bat to use -Xbootclasspath/a: instead of -cp for %_TOOL_CLASSPATH%. However looking into the usebootcp in the *nix script I can see following comment:

# default to the boot classpath for speed, except on cygwin/mingw/msys because
# JLine on Windows requires a custom DLL to be loaded.
unset usebootcp
if [[ -z "$cygwin$mingw$msys" ]]; then
  usebootcp="true"
fi

So I suspect that if you want to use scala.bat for REPL, moving all Scala libs to the Boot Classpath might be a bad idea. If this is the case, you probably need to create a copy of scala.bat (such as scala_run_script.bat) change it and use it to run your Scala scripts leaving standard scala.bat for REPL.