Do Layer Normalization in Pytorch without learnable parameters?

Question

We can add layer normalization in Pytorch by doing: torch.nn.LayerNorm(shape). However, this is layer normalization with learnable parameters. I.e, it's the following equation:

Does Pytorch have builtin layer normalization without learnable parameters?

Answer 1

You can use nn.LayerNorm, setting the elementwise flag to False. This way the layer won't have learnt parameters. See source code for additional reference.