Menu
How can I create the sparksession?
scala> import org.apache.spark.SparkConf
import org.apache.spark.SparkConf
scala> import org.apache.spark.SparkContext
import org.apache.spark.SparkContext
scala> val conf = SparkSession.builder.master("local").appName("testing").enableHiveSupport().getOrCreate()
<console>:27: error: not found: value SparkSession
val conf = SparkSession.builder.master("local").appName("testing").enableHiveSupport().getOrCreate()
403
SparkSession Encapsulates SparkContextIt allows you to configure Spark configuration parameters. And through SparkContext, the driver can access other contexts such as SQLContext, HiveContext, and StreamingContext to program Spark.
SparkSession from DataFrame If you have a DataFrame, you can use it to access the SparkSession, but it's best to just grab the SparkSession with getActiveSession() . Let's shut down the active SparkSession to demonstrate the getActiveSession() returns None when no session exists.
SparkSession is available in spark 2.x
import org.apache.spark.sql.SparkSession
Though when you start spark shell SparkSession is already available as spark variable.
174
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
