I am not sure how to launch a Java app with spark-submit
.
When I run the following command:
spark@mcava-master:/home/miren/NetBeansProjects$ /opt/spark/bin/spark-submit --class /home/miren/NetBeansProjects/SparkExample/src/main/java/com/mycompany/sparkexample/CountWords.java --master spark://192.168.1.105:7077 /home/miren/NetBeansProjects/SparkExample/target/SparkExample-1.0-SNAPSHOT.jar spark://192.168.1.105:7077
I get an exception:
java.lang.ClassNotFoundException: /home/miren/NetBeansProjects/SparkExample/src/main/java/com/mycompany/sparkexample/CountWords.java
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:174)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:689)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
How do I indicate the classpath?
You need to call spark-submit
in the following way:
spark-submit --class com.mycompany.sparkexample.CountWords
--master spark://192.168.1.105:7077
/home/miren/NetBeansProjects/SparkExample/target/SparkExample-1.0-SNAPSHOT.jar
{your-app-arguments-here}
The --class
parameter takes the fully qualified name of the object containing your main method.
The documentation contains more information about using spark-submit
properly.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With