How do I define my own operators in TensorFlow

Question

In TensorFlow, we can use tf.nn.l2_loss() for doing L2 regularization. Let's say I want to define my own regularization operator for L1 regularization (call it tf.nn.l1_loss()). How would I go about it? I am having a hard time locating operator definitions in the TensorFlow source code.

Answer 1

As the comment suggested, there is a how-to guide for adding an op to TensorFlow. This guide covers adding a new op that is implemented in C++. In general, you should do this in the following situations:

• The op cannot be implemented using existing TensorFlow ops (for example, l1_loss could be implemented using the existing element-wise and reduction operators as a Python function).
• A C++ implementation is necessary for performance (or memory consumption) reasons.
• The op could be implemented as a composition of ops, but it has a gradient that can be computed more efficiently (or with better numerical stability) than computing the gradients op-by-op. (This is why tf.nn.l2_loss is implemented as a fused op in C++.)