It is possible to start an embedded instance of apache Spark node?

Question

I want to start an instance of a standalone Apache Spark cluster embedded into my java app. I tried to find some documentation at their website but not look yet.

Is this possible?

Answer 1

You can create SparkContext in local mode, you just need to provide "local" as a spark master url to SparkConf

val sparkConf = new SparkConf().
  setMaster("local[2]").
  setAppName("MySparkApp")

val sc = new SparkContext(sparkConf)

Answer 2

Yes -- you can use Spark in an embedded way with a "local" master.

SparkConf sparkConf = new SparkConf();//Create new spark config
sparkConf.setMaster("local[8]"); // local, using 8 cores (you can vary the number)
sparkConf.setAppName("MyApp");
SparkContext sc = new SparkContext(sparkConf);

This will run Spark within your JVM.