Official comment shows that "This has any effect only on modules such as Dropout or BatchNorm." But I don't understand its implementation.

Dropout and BatchNorm (and maybe some custom modules) behave differently during training and evaluation. You must let the model know when to switch to eval mode by calling <code>.eval()</code> on the model. This sets <code>self.training</code> to <code>False</code> for every module in the model. If you are implementing your own module that must behave differently during training and evaluation, you can check the value of <code>self.training</code> while doing so.

What's the meaning of function eval() in torch.nn module

1 Answers

Dropout and BatchNorm (and maybe some custom modules) behave differently during training and evaluation. You must let the model know when to switch to eval mode by calling .eval() on the model.

This sets self.training to False for every module in the model. If you are implementing your own module that must behave differently during training and evaluation, you can check the value of self.training while doing so.