Spark SQL "<=>" operator

Question

I recently came across this operator: "<=>" in a Scala Spark project. I am having trouble finding any documentation on it. Does any know how it works, or know of some documentation?

I believe it has something to with equality operations.

Thanks

Observation 1 (2/12/2020) | Spark 2.3.2 I have done some experimentation and noticed that performing joins on co-located data with "<=>" causes a shuffle to happen, while "==" does not.

Answer 1

Checkout the Section "Supported Hive Feature on Spark SQL Programming guide link and you will find it in the list of Hive Operators supported by Spark.

Here is what it does:

• Returns same result with EQUAL(=) operator for non-null operands

however:

• it returns TRUE if both are NULL

• it returns FALSE if one of them is NULL