How to write a PyTorch sequential model?

Question

So far, I wrote my MLP, RNN and CNN in Keras, but now PyTorch is gaining popularity inside deep learning communities, and so I also started to learn this framework. I am a big fan of sequential models in Keras, which allow us to make simple models very fast. I also saw that PyTorch has this functionality, but I don't know how to code one. I tried this way

import torch
import torch.nn as nn

net = nn.Sequential()
net.add(nn.Linear(3, 4))
net.add(nn.Sigmoid())
net.add(nn.Linear(4, 1))
net.add(nn.Sigmoid())
net.float()

print(net)

but it is giving this error

AttributeError: 'Sequential' object has no attribute 'add'

Also, if possible, can you give simple examples for RNN and CNN models in PyTorch sequential model?

Answer 1

Sequential does not have an add method at the moment, though there is some debate about adding this functionality.

As you can read in the documentation nn.Sequential takes as argument the layers separeted as sequence of arguments or an OrderedDict.

If you have a model with lots of layers, you can create a list first and then use the * operator to expand the list into positional arguments, like this:

layers = []
layers.append(nn.Linear(3, 4))
layers.append(nn.Sigmoid())
layers.append(nn.Linear(4, 1))
layers.append(nn.Sigmoid())

net = nn.Sequential(*layers)

This will result in a similar structure of your code, as adding directly.

Answer 2

As described by the correct answer, this is what it would look as a sequence of arguments:

device = torch.device('cpu')
if torch.cuda.is_available():
    device = torch.device('cuda')

net = nn.Sequential(
      nn.Linear(3, 4),
      nn.Sigmoid(),
      nn.Linear(4, 1),
      nn.Sigmoid()
      ).to(device)


print(net)

Sequential(
  (0): Linear(in_features=3, out_features=4, bias=True)
  (1): Sigmoid()
  (2): Linear(in_features=4, out_features=1, bias=True)
  (3): Sigmoid()
  )