TensorFlow operator overloading

Question

What is the difference between

   tf.add(x, y) 

and

   x + y 

in TensorFlow? What would be different in your computation graph when you construct your graph with + instead of tf.add()?

More generally, are + or other operations overloaded for tensors?

Answer 1

If at least one of x or y is a tf.Tensor object, the expressions tf.add(x, y) and x + y are equivalent. The main reason you might use tf.add() is to specify an explicit name keyword argument for the created op, which is not possible with the overloaded operator version.

Note that if neither x nor y is a tf.Tensor—for example if they are NumPy arrays—then x + y will not create a TensorFlow op. tf.add() always creates a TensorFlow op and converts its arguments to tf.Tensor objects. Therefore, if you are writing a library function that might accept both tensors and NumPy arrays, you might prefer to use tf.add().

The following operators are overloaded in the TensorFlow Python API:

• __neg__ (unary -)
• __abs__ (abs())
• __invert__ (unary ~)
• __add__ (binary +)
• __sub__ (binary -)
• __mul__ (binary elementwise *)
• __div__ (binary / in Python 2)
• __floordiv__ (binary // in Python 3)
• __truediv__ (binary / in Python 3)
• __mod__ (binary %)
• __pow__ (binary **)
• __and__ (binary &)
• __or__ (binary |)
• __xor__ (binary ^)
• __lt__ (binary <)
• __le__ (binary <=)
• __gt__ (binary >)
• __ge__ (binary >=)

Please note, __eq__ ( binary == ) is not overloaded. x == y will simply return a Python boolean whether x and y refer to the same tensor. You need to use tf.equal() explicitly to check for element-wise equality. Same goes for not equal, __ne__ ( binary != ).