Menu
I want to design a convolutional neural network which occupy GPU resource no more than Alexnet.I want to use FLOPs to measure it but I don't know how to calculate it.Is there any tools to do it,please?
939
Calculating the FLOPs in a Model ➡️ For a refresher on CNNs, you can check this cheatsheet. To calculate the FLOPs in a model, here are the rules: Convolutions - FLOPs = 2x Number of Kernel x Kernel Shape x Output Shape. Fully Connected Layers - FLOPs = 2x Input Size x Output Size.
So when we calculate FLOPs, we only need to multiply the size of the feature map on the basis of the parameters, that is, for a certain convolutional layer, the number of FLOPs is: [(K_h * K_w )* C_{in} + 1][(H_{out}W_{out})* C_{out} ] = [(K_h * K_w * C_{in}) * C_{out} + C_{out}][H_{out}W_{out}]= num_{parameter}*size_{ ...
It is usually calculated using the number of multiply-add operations that a model performs. Multiply-add operations, as the name suggests, are operations involving multiplication and addition of 2 or more variables. For example, the expression, a * b + c * d, has 2 flops while a * b + c * d + e * f + f * h has 4 flops.
To calculate the learnable parameters here, all we have to do is just multiply the by the shape of width m, height n, previous layer's filters d and account for all such filters k in the current layer. Don't forget the bias term for each of the filter.
For online tool see http://dgschwend.github.io/netscope/#/editor . For alexnet see http://dgschwend.github.io/netscope/#/preset/alexnet . This supports most wide known layers. For custom layers you will have to calculate yourself.
86
For future visitors, if you use Keras and TensorFlow as Backend then you can try the following example. It calculates the FLOPs for the MobileNet.
import tensorflow as tf
import keras.backend as K
from keras.applications.mobilenet import MobileNet
run_meta = tf.RunMetadata()
with tf.Session(graph=tf.Graph()) as sess:
K.set_session(sess)
net = MobileNet(alpha=.75, input_tensor=tf.placeholder('float32', shape=(1,32,32,3)))
opts = tf.profiler.ProfileOptionBuilder.float_operation()
flops = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)
opts = tf.profiler.ProfileOptionBuilder.trainable_variables_parameter()
params = tf.profiler.profile(sess.graph, run_meta=run_meta, cmd='op', options=opts)
print("{:,} --- {:,}".format(flops.total_float_ops, params.total_parameters))
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
