Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #2 'weight'

Question

Firstly I have used like 'model.cuda()' to convert model and data to cuda. But it still has such a problem. I debug every layer of the model, and weights for every module has iscuda=True. So does anyone know why there is such a problem?

I have two models, one is resnet50 and another one which contains the first one as backbone.

class FC_Resnet(nn.Module):
    def __init__(self, model, num_classes):
        super(FC_Resnet, self).__init__()

        # feature encoding
        self.features = nn.Sequential(
            model.conv1,
            model.bn1,
            model.relu,
            model.maxpool,
            model.layer1,
            model.layer2,
            model.layer3,
            model.layer4)

        # classifier
        num_features = model.layer4[1].conv1.in_channels
        self.classifier = nn.Sequential(
            nn.Conv2d(num_features, num_classes, kernel_size=1, bias=True))

    def forward(self, x):
        # children=self.features.children()
        # for child in children:
        #     if child.weight is not None:
        #         print(child.weight.device)
        x = self.features(x)
        x = self.classifier(x)
        return x

def fc_resnet50(num_classes=20, pre_trained=True):
    model = FC_Resnet(models.resnet50(pre_trained), num_classes)

    return model

And another one:

class PeakResponseMapping(nn.Sequential):
    def __init__(self, *args, **kargs):
        super(PeakResponseMapping, self).__init__(*args)
        ...

    def forward(self, input, class_threshold=0, peak_threshold=30, retrieval_cfg=None):
        assert input.dim() == 4
        if self.inferencing:
            input.requires_grad_()

        class_response_maps = super(PeakResponseMapping, self).forward(input)

        return class_response_maps

And the main is very simple:

def main():
    dataset = VOC(img_transform=image_transform())
    dataloader = DataLoader(dataset, batch_size=BATCH_SIZE, shuffle=True)

    model = peak_response_mapping(fc_resnet50(), win_size=3, sub_pixel_locating_factor=8, enable_peak_stimulation=True)
    model=model.cuda()

    for step, (b_x, b_y) in enumerate(dataloader):
        b_x.cuda()
        b_y.cuda()

        result = model.forward(b_x)

Answer 1

Somewhere down in the stack trace, Torch is expecting a CPU tensor (torch.FloatTensor) but is getting a GPU / CUDA tensor (torch.cuda.FloatTensor).

Given a tensor tensor:

• tensor.to('cpu') returns the CPU version of the tensor
• tensor.to('cuda') returns the CUDA version of the tensor

To write hardware-agnostic code:

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

Then you can do:

tensor.to(device)

For the OP, this becomes:

result = model.forward(b_x.to(device))

Answer 2

You need to assign b_x.cuda() back to b_x:

b_x = b_x.cuda()
b_y = b_y.cuda()

Looking at the documentation of .cuda():

Returns a copy of this object in CUDA memory.

so, b_x.cuda() returns a copy of b_x and does not affect b_x in an in-place manner.