How to multiply a matrix by a vector in PyTorch

Question

I'm playing around with PyTorch with the aim of learning it, and I have a very dumb question: how can I multiply a matrix by a single vector?

Here's what I've tried:

>>> import torch
>>> a = torch.rand(4,4)
>>> a

 0.3162  0.4434  0.9318  0.8752
 0.0129  0.8609  0.6402  0.2396
 0.5720  0.7262  0.7443  0.0425
 0.4561  0.1725  0.4390  0.8770
[torch.FloatTensor of size 4x4]

>>> b = torch.rand(4)
>>> b

 0.1813
 0.7090
 0.0329
 0.7591
[torch.FloatTensor of size 4]

>>> a.mm(b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: invalid argument 2: dimension 1 out of range of 1D tensor at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensor.c:24
>>> a.mm(b.t())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: t() expects a 2D tensor, but self is 1D
>>> b.mm(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: matrices expected, got 1D, 2D tensors at /Users/soumith/code/builder/wheel/pytorch-src/torch/lib/TH/generic/THTensorMath.c:1288
>>> b.t().mm(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: t() expects a 2D tensor, but self is 1D

On the other hand, if I do

>>> b = torch.rand(4,2)

then my first attempt, a.mm(b), works fine. So the problem is just that I'm multiplying a vector rather than a matrix --- but how can I do this?

Answer 1

You're looking for

torch.mv(a,b)

Note that for the future, you may also find torch.matmul() useful. torch.matmul() infers the dimensionality of your arguments and accordingly performs either dot products between vectors, matrix-vector or vector-matrix multiplication, matrix multiplication or batch matrix multiplication for higher order tensors.

Answer 2

This is a self-answer to supplement @mexmex's correct and useful answer.

In PyTorch, unlike numpy, 1D Tensors are not interchangeable with 1xN or Nx1 tensors. If I replace

>>> b = torch.rand(4)

with

>>> b = torch.rand((4,1))

then I will have a column vector, and matrix multiplication with mm will work as expected.

But this is not necessary, because as @mexmex points out there is an mv function for matrix-vector multiplication, as well as a matmul function that dispatches the appropriate function depending on the dimensions of its input.