Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spark Implicit $ for DataFrame

Where in the sqlContext.implicits._ does it define the $"string" to represent a dataframe call to the parent dataframe's column? Specifically I was confused on seeing something like the following:

import sqlContext.implicits._
df.where($"type".isin("type1","type2") and $"status".isin("completed","inprogress"))
like image 207
Ben Nelson Avatar asked Feb 13 '17 21:02

Ben Nelson


1 Answers

If you see the following link

class SQLContext.implicits$.StringToColumn

Converts $"col name" into an Column.

Here is the link for latest version.

like image 96
user2844511 Avatar answered Oct 22 '22 00:10

user2844511