converting list of tensors to tensors pytorch

Question

I have list of tensor where each tensor has a different size. How can I convert this list of tensors into a tensor using PyTorch?

For instance,

x[0].size() == torch.Size([4, 8])
x[1].size() == torch.Size([4, 7])  # different shapes!

This:

torch.tensor(x)

Gives the error:

ValueError: only one element tensors can be converted to Python scalars

Answer 1

You might be looking for cat.

However, tensors cannot hold variable length data.

for example, here we have a list with two tensors that have different sizes(in their last dim(dim=2)) and we want to create a larger tensor consisting of both of them, so we can use cat and create a larger tensor containing both of their data.

also note that you can't use cat with half tensors on cpu as of right now so you should convert them to float, do the concatenation and then convert back to half

import torch

a = torch.arange(8).reshape(2, 2, 2)
b = torch.arange(12).reshape(2, 2, 3)
my_list = [a, b]
my_tensor = torch.cat([a, b], dim=2)
print(my_tensor.shape) #torch.Size([2, 2, 5])

you haven't explained your goal so another option is to use pad_sequence like this:

from torch.nn.utils.rnn import pad_sequence
a = torch.ones(25, 300)
b = torch.ones(22, 300)
c = torch.ones(15, 300)
pad_sequence([a, b, c]).size() #torch.Size([25, 3, 300])

edit: in this particular case, you can use torch.cat([x.float() for x in sequence], dim=1).half()

Answer 2

Tensor in pytorch isn't like List in python, which could hold variable length of objects.

In pytorch, you can transfer a fixed length array to Tensor:

>>> torch.Tensor([[1, 2], [3, 4]])
>>> tensor([[1., 2.],
            [3., 4.]])

Rather than:

>>> torch.Tensor([[1, 2], [3, 4, 5]])
>>> 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-809c707011cc> in <module>
----> 1 torch.Tensor([[1, 2], [3, 4, 5]])

ValueError: expected sequence of length 2 at dim 1 (got 3)

And it's same to torch.stack.