Concatenate Two Tensors in Pytorch

Question

How do I pad a tensor of shape [71 32 1] with zero vectors to make it [100 32 1]?

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 2. Got 32 and 71 in dimension 0 at /pytorch/aten/src/THC/generic/THCTensorMath.cu:87

I tried by concatenating a padding vector of zeros of shape [29 32 1]. I get the error above. I try with a padding vector of zeros of shape [29 32 1], I still get an error.

Answer 1

In order to help you better, you need to post the code that caused the error, without it we are just guessing here...

Guessing from the error message you got:

1.

Sizes of tensors must match except in dimension 2

pytorch tries to concat along the 2nd dimension, whereas you try to concat along the first.

2.

Got 32 and 71 in dimension 0

It seems like the dimensions of the tensor you want to concat are not as you expect, you have one with size (72, ...) while the other is (32, ...).
You need to check this as well.

Working code

Here's an example of concat

import torch

x = torch.rand((71, 32, 1))
# x.shape = torch.Size([71, 32, 1])
px = torch.cat((torch.zeros(29, 32, 1, dtype=x.dtype, device=x.device), x), dim=0)
# px.shape = torch.Size([100, 32, 1])

Alternatively, you can use functional.pad:

from torch.nn import functional as F

px = F.pad(x, (0, 0, 0, 0, 29, 0))