How can spark-shell work without installing Scala beforehand?

Question

I have downloaded Spark 1.2.0 (pre-built for Hadoop 2.4). In its quick start doc, it says:

It is available in either Scala or Python.

What confuses me is that my computer doesn't have Scala installed separately before (OS X 10.10), but when I type spark-shell it works well, and the output shows:

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_25)

as depicted in the screenshot:

I didn't install any Scala distribution before.

How can spark-shell run without Scala?

Answer 1

tl;dr Scala binaries are included in Spark already (to make Spark users' life easier).

Under Downloading in Spark Overview you can read about what is required to run Spark:

Spark runs on both Windows and UNIX-like systems (e.g. Linux, Mac OS). It’s easy to run locally on one machine — all you need is to have java installed on your system PATH, or the JAVA_HOME environment variable pointing to a Java installation.

Spark runs on Java 6+ and Python 2.6+. For the Scala API, Spark 1.2.0 uses Scala 2.10. You will need to use a compatible Scala version (2.10.x).

Answer 2

Scala program, including spark-shell, is compiled to Java byte code, which can be run with Java virtual machine (JVM). Therefore, as long as you have JVM installed, meaning java command, you can run the Spark-related tools written in Scala.