Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

what is difference between SparkSession and SparkContext? [duplicate]

I only know the version difference but do not know the functionality or else. i.e Sparksession has internally sparkcontext and conf.

like image 672
gopal kulkarni Avatar asked Mar 30 '18 12:03

gopal kulkarni


1 Answers

In older version of Spark there was different contexts that was entrypoints to the different api (sparkcontext for the core api, sql context for the spark-sql api, streaming context for the Dstream api etc...) this was source of confusion for the developer and was a point of optimization for the spark team, so in the most recent version of spark there is only one entrypoint (the spark session) and from this you can get the various other entrypoint (the spark context , the streaming context , etc ....)

like image 164
eugenio calabrese Avatar answered Oct 11 '22 00:10

eugenio calabrese