PyTorch: How do the means and stds get calculated in the Transfer Learning tutorial?

Question

I'm going through the PyTorch Transfer Learning tutorial at: link

In the data augmentation stage, there is the following step to normalize images:

transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])

I can understand why it's doing this but I can't find how the mean and std values get calculated? I tried to calculate the mean on the train data set and the mean values are:

array([ 0.11727478,  0.04542569, -0.28624609], dtype=float32)

Answer 1

Your numbers don't seem right to me; since the ToTensor transform has output in the range [0.0, 1.0] it shouldn't be possible to get a negative mean.

If I calculate the mean with

traindata = datasets.ImageFolder(data_dir + '/train', transforms.ToTensor())
image_means = torch.stack([t.mean(1).mean(1) for t, c in traindata])
image_means.mean(0)

I get (0.5143, 0.4760, 0.3487) and for the validation set (0.5224, 0.4799, 0.3564). These are closer to the numbers in the tutorial. Searching for the specific numbers, you'll see that they appear in the Imagenet example, so my guess is that they are the means of the Imagenet dataset, of which the tutorial dataset is a subset.