How to count Multiply-Adds operations?

Question

Given a defined Convolutional Neural Network, is there a function or method on how to compute the number of Multiply-Adds operations?

The term was introduced MobileNetV2 paper : https://arxiv.org/pdf/1801.04381.pdf.

We evaluate the trade-offs between accuracy, and number of operations measured by multiply-adds (MAdd), as well as actual latency, and the number of parameters.

Answer 1

Counting the Multiply-Add operations is equivalent to calculating the FLOPs of a model. This can be achieved using the profiler from tensorflow.

flops = tf.profiler.profile(graph,\
     options=tf.profiler.ProfileOptionBuilder.float_operation())
print('FLOP = ', flops.total_float_ops)

Be sure to look at the caveats explained in this answer. Basically:

• The number of FLOPs calculated might include initialisation operations like Multiply-Add from initialising your weights with a Gaussian distribution for instance, you need to freeze the graph to rule those possibly irrelevant Multiply-Add operations,
• The Multiply-Add calculations from TensorFlow are approximate. An issue has been opened on this matter here.