Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Difference between `tf.nn.batch_normalization` and `tf.nn.fused_batch_norm`?

Tags:

tensorflow

Their documentation is short and they both refer to the same paper. Is there a difference in what those two functions implement? If no, is one of them soon-to-be obsolete by the other, which one of the two is recommended for use?

like image 249
P-Gn Avatar asked Mar 09 '23 02:03

P-Gn


1 Answers

According to the performance guide

The non-fused batch norm does computations using several individual Ops. Fused batch norm combines the individual operations into a single kernel, which runs faster.

EDIT: 1/6/2020

The original link no longer works. This is a web archive link provided by Rika. The updated text says:

Fused batch norm combines the multiple operations needed to do batch normalization into a single kernel. Batch norm is an expensive process that for some models makes up a large percentage of the operation time. Using fused batch norm can result in a 12%-30% speedup.

like image 72
Yahia Zakaria Avatar answered May 19 '23 15:05

Yahia Zakaria