What exactly is the definition of a 'Module' in PyTorch?

Question

Please excuse the novice question, but is Module just the same as saying model?

That's what it sounds like, when the documentation says:

Whenever you want a model more complex than a simple sequence of existing Modules you will need to define your model (as a custom Module subclass).

Or... when they mention Module, are they referring to something more formal and computer-sciency, like a protocol / interface type thing?

Answer 1

It's a simple container.

From the docs of nn.Module

Base class for all neural network modules. Your models should also subclass this class. Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes. Submodules assigned in this way will be registered, and will have their parameters converted too when you call .cuda(), etc.

From the tutorial:

All network components should inherit from nn.Module and override the forward() method. That is about it, as far as the boilerplate is concerned. Inheriting from nn.Module provides functionality to your component. For example, it makes it keep track of its trainable parameters, you can swap it between CPU and GPU with the .to(device) method, where device can be a CPU device torch.device("cpu") or CUDA device torch.device("cuda:0").

A module is a container from which layers, model subparts (e.g. BasicBlock in resnet in torchvision) and models should inherit. Why should they? Because the inheritance from nn.Module allows you to call methods like to("cuda:0"), .eval(), .parameters() or register hooks easily.

• why not just call the 'module' a model, and call the layers 'layers'? I suppose maybe it's just semantics and splitting hairs, but still...

That's an API design choice and I find having only a Module class instead of two separate Model and Layers to be cleaner and to allow more freedom (it's easier to send just a part of the model to GPU, to get parameters only for some layers...).