What is a buffer in Pytorch?

Question

I understand what register_buffer does and the difference between register_buffer and register_parameters.

But what is the precise definition of a buffer in PyTorch?

Answer 1

This can be answered looking at the implementation:

def register_buffer(self, name, tensor):
    if '_buffers' not in self.__dict__:
        raise AttributeError(
            "cannot assign buffer before Module.__init__() call")
    elif not isinstance(name, torch._six.string_classes):
        raise TypeError("buffer name should be a string. "
                        "Got {}".format(torch.typename(name)))
    elif '.' in name:
        raise KeyError("buffer name can't contain \".\"")
    elif name == '':
        raise KeyError("buffer name can't be empty string \"\"")
    elif hasattr(self, name) and name not in self._buffers:
        raise KeyError("attribute '{}' already exists".format(name))
    elif tensor is not None and not isinstance(tensor, torch.Tensor):
        raise TypeError("cannot assign '{}' object to buffer '{}' "
                        "(torch Tensor or None required)"
                        .format(torch.typename(tensor), name))
    else:
        self._buffers[name] = tensor

That is, the buffer's name:

• must be a string: not isinstance(name, torch._six.string_classes)
• cannot contain a . (dot): '.' in name
• cannot be an empty string: name == ''
• cannot be an attribute of the Module: hasattr(self, name)
• should be unique: name not in self._buffers

and the tensor (guess what?):

• should be a Tensor: isinstance(tensor, torch.Tensor)

So, the buffer is just a tensor with these properties, registered in the _buffers attribute of a Module;