Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Difference between == and === in Scala, Spark

I am from a Java background and new to Scala.

I am using Scala and Spark. But I'm not able to understand where I use ==and ===.

Could anyone let me know in which scenario I need to use these two operators, and what's are difference between == and ===?

like image 388
Avijit Avatar asked Sep 14 '16 12:09

Avijit


People also ask

Why === is used in Spark Scala?

The triple equals operator === is normally the Scala type-safe equals operator, analogous to the one in Javascript. Spark overrides this with a method in Column to create a new Column object that compares the Column to the left with the object on the right, returning a boolean.

What is the difference between equals () and ==? Scala?

== is a final method, and calls . equals , which is not final. This is radically different than Java, where == is an operator rather than a method and strictly compares reference equality for objects.

What is == in Scala?

In Java, C++, and C# the == operator tests for reference, not value equality. In contrast, Ruby's == operator tests for value equality. But in Scala, == is testing for value equality.

Is PySpark slower than Scala?

Scala definitely offers better performance than Python but it is not always 10x faster, as the number of cores increases, the performance advantage of Scala starts to dwindle. Deciding on Scala vs Python for Spark depends on the features that best fit the project needs as each one has its own pros and cons.


3 Answers

The "==" is using the equals methods which checks if the two references point to the same object. The definition of "===" depends on the context/object. For Spark , "===" is using the equalTo method. See

  • for == https://spark.apache.org/docs/2.0.0/api/java/org/apache/spark/sql/Column.html#equals(java.lang.Object)
  • for === https://spark.apache.org/docs/2.0.0/api/java/org/apache/spark/sql/Column.html#equalTo(java.lang.Object)

(Since you are referencing Spark:) An important difference for Spark is the return value. For Column:

  • == returns a boolean

  • === returns a column (which contains the result of the comparisons of the elements of two columns)

like image 199
Christian Fries Avatar answered Oct 23 '22 10:10

Christian Fries


Generally speaking, they are just functions.

For different types, "==" and "===" might be defined or "overloaded" for different meanings.

For example, in some test framework, "===" is defined for some special function. See this.

like image 14
Lifu Huang Avatar answered Oct 23 '22 11:10

Lifu Huang


ScalaTest lets you use Scala's assertion syntax, but defines a triple equals operator (===) to give you better error messages. The following code would give you an error indicating only that an assertion failed:

assert(1 == 2) Using triple equals instead would give you the more informative error message, "1 did not equal 2":

assert(1 === 2)

have a look at this page for more details; What is the === (triple-equals) operator in Scala Koans?

like image 5
Aditya Agarwal Avatar answered Oct 23 '22 11:10

Aditya Agarwal