Spark-submit class not found exception

Question

I am trying to run this simple spark application with the spark submit command using this quick start tutorial. http://spark.apache.org/docs/1.2.0/quick-start.html#self-contained-applications. when I try to run it using spark-1.4.0-bin-hadoop2.6\bin>spark-submit --class " SimpleApp" --master local[4] C:/.../Documents/Sparkapp/target/scala- 2.10/simple-project_2.10-1.0.jar I get the following exception:

java.lang.ClassNotFoundException: SimpleApp
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSub
mit$$runMain(SparkSubmit.scala:633)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:16
9)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/06/17 09:45:11 INFO Utils: Shutdown hook called

Does anyone know how to fix this? Any help is greatly appreciated.

Answer 1

The thing is that we have to submit the class file to the spark cluster whom we want to execute or will take use as a supporting file, so follow these steps -

1. Create a jar file of this class -> In eclipse you can export this class as a jar file. -> From command line

   jar cf myJar.jar myClass.class

2. Edit your code as follows -

   val jarFile = "SimpleApp.jar"; #path

   val conf = new SparkConf().setAppName("Simple Application").setJars(Array(jarFile));

Answer 2

Simple solution to this is check your full name of the class in my case it was main.java.SimpleApp and it worked. Thanks Svend for pointing out that