Adding modules in Pytorch Custom Module

Question

Is it considered bad practice to add modules to a custom pytorch nn.Module using self.add_module()? All of the documentation seems to assign the layers to properties, then access them in the forward() method.

For example:

class ConvLayer(nn.Module):
    def __init__(self):
        self.add_module('conv',nn.Conv2d(...))
        self.add_module('bn',nn.BatchNorm2d(...))

Answer 1

Calling add_module will add an entry to the _modules dict. The Module class also overwrites __getattr__ so that when you try to access a layer, it will look inside the _modules dict, despite the fact that the layer is not actually an attribute of the object. But from the user's perspective, it doesn't make a difference whether module.layer returns an actual property or simply an element from some private dict.