Good Practice to insure only one Spark context in application

Question

I am looking for a good way to insure that my app is only using one single Spark Context (sc). While developing I often run into errors and have to restart my Play! server to re test my modifications. Would a Singleton pattern be solution ?

object sparckContextSingleton {
  @transient private var instance: SparkContext = _
  private val conf : SparkConf = new SparkConf()
    .setMaster("local[2]")
    .setAppName("myApp")

  def getInstance(): SparkContext = {
    if (instance == null){
      instance = new SparkContext(conf)
    }
    instance
  }  
}

This does not make a good job. Should I stop the SparkContext?

Answer 1

This should be enough to do the trick, important is to use val and not var.

object SparkContextKeeper {
  val conf = new SparkConf().setAppName("SparkApp")
  val context= new SparkContext(conf)
  val sqlContext = new SQLContext(context)
}