expected Double tensor (got Float tensor) in Pytorch

Question

I want to create nn.Module in Pytorch. I used following code for text related problem (In fact I use Glove 300d pre-trained embedding and weighted average of words in a sentence to do classification).

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(300, 128, kernel_size=5)
        self.conv2 = nn.Conv1d(128, 64, kernel_size=2)
        self.conv2_drop = nn.Dropout()
        self.fc1 = nn.Linear(64, 20)
        self.fc2 = nn.Linear(20, 2)

    def forward(self, x):
        x = F.relu(F.avg_pool1d(self.conv1(x), 2))
        x = F.relu(F.avg_pool1d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 1)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        return self.fc2(x)

But it gives me following error:

Traceback (most recent call last):
    x = F.relu(F.avg_pool1d(self.conv1(x), 2))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/torch/nn/modules/conv.py", line 154, in forward
    self.padding, self.dilation, self.groups)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/torch/nn/functional.py", line 83, in conv1d
    return f(input, weight, bias)
RuntimeError: expected Double tensor (got Float tensor)

I fairly new to Conv1d, and most of tutorial used Conv1d for image problems. Can anybody give me some idea what's the problem?

I also added model.double() inside forward method but gives me another error:

RuntimeError: Given input size: (300 x 1 x 1). Calculated output size: (128 x 1 x -3). Output size is too small

Answer 1

Error 1

RuntimeError: expected Double tensor (got Float tensor)

This happens when you pass a double tensor to the first conv1d function. Conv1d only works with float tensor. Either do,

1. conv1.double() , or
2. model.double().

Which is what you have done, and that is correct.

Error 2

RuntimeError: Given input size: (300 x 1 x 1). Calculated output size: (128 x 1 x -3). Output size is too small

This is because you are passing an input to which a convolution with a window size of 5 is not valid. You will have to add padding to your Conv1ds for this to work, like so:

self.conv1 = nn.Conv1d(300, 128, kernel_size=5, padding=2)

If you dont wish to add padding, then given (batch_size, in_channels, inp_size) as the size of the input tensor, you have to make sure that your inp_size is greater than 5.

All fixes combined

Make sure that your sizes are correct for the rest of your network. Like so:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv1d(300, 128, kernel_size=5, padding=2)
        self.conv2 = nn.Conv1d(128, 64, kernel_size=2, padding=1)
        self.conv2_drop = nn.Dropout()
        self.fc1 = nn.Linear(64, 20)
        self.fc2 = nn.Linear(20, 2)

    def forward(self, x):
        x = F.relu(F.avg_pool1d(self.conv1(x), 2, padding=1))
        x = F.relu(F.avg_pool1d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(1, -1) # bonus fix, Linear needs (batch_size, in_features) and not (in_features, batch_size) as input.
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        return self.fc2(x)

if __name__ == '__main__':

    t = Variable(torch.randn((1, 300, 1))).double() # t is a double tensor
    model = Net()
    model.double() # this will make sure that conv1d will process double tensor
    out = model(t)