how does BatchNorm1d() method whithin the torch library work?

Question

I'm learning pytorch, I don;t know if this question is stupid but I can't find the official web for explaining nn.batchnorm1d. I'm wondering how torch.nn.BatchNorm1d(d1) work? I know that batch norm is about making mean and variance of a batch of example to be 0 and 1 respectively. I'm wondering if there is nn.batchnorm2d, if so, what does it do? what is the d1 parameter ?

Answer 1

BatchNorm1d normalises data to 0 mean and unit variance for 2/3-dimensional data (N, C) or (N, C, L), computed over the channel dimension at each (N, L) or (N,) slice; while BatchNorm2d does the same thing for 4 dimensions (N, C, H, W), computed over the channel dimension at each (N, H, W) slice.

Which one to use depends on the dimensionality of input data. For instance in image processing, feature maps ususally have 2 spatial dimensions (N, C, H, W), so BatchNorm2d is useful here. However for some NLP tasks, if there is only the length dimension to consider, one would use BatchNorm1d. For both functions, the d1 parameter is the number of features, and equals dim C of the input tensor.