What is the =!= operator in Scala?

Question

I've come across this operator in a Spark application written in Scala and would like to understand it. Example...

val filtered = df
    .filter(lower('entry) =!= "blah blah")

Thanks.

Answer 1

That's just a method name like any other method name. It has no special meaning whatsoever.

It is also not a well-known method name in Scala. It seems to come from some library; you need to look at the documentation of whatever library you are using to figure out what it does.

In this case, it appears to be org.apache.spark.sql.Column.=!=:

def =!=(other: Any): Column

Inequality test.

// Scala:
df.select( df("colA") =!= df("colB") )
df.select( !(df("colA") === df("colB")) )

// Java:
import static org.apache.spark.sql.functions.*;
df.filter( col("colA").notEqual(col("colB")) );

Answer 2

you must use the =!= operator so that you don’t just compare the unevaluated column expression to a string but instead to the evaluated one

source:spark-the definitive guide

Answer 3

That is this method in org.apache.spark.sql.Column which serves as an inequality test.