Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Passing sqlContext as an implicit parameter in Spark

I have a function in a Scala object which has the following signature

def f(v1:Int)(implicit sqlContext: SQLContext)

when I try to call this function from spark-shell I call it like

f(1)

and I expect the existing sqlContext gets passed to it implicitly but it doesn't. How can I make it to work so the sqlContext get passed to this function automatically?

--------------update-------------------

I tried to import sqlContext.implicits._ in the spark-shell before calling my function but it didn't help

like image 638
HHH Avatar asked May 12 '26 22:05

HHH


1 Answers

You just need to add a SQLContext implicitly to the same context which you are calling your function:

implicit val sqlContext = new SQLContext() // just an example
// and then
f(1)

If you are using apacha Spark, you can use this import:

import sqlContext.implicits._
like image 145
Juliano Alves Avatar answered May 16 '26 13:05

Juliano Alves



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!