Adding arrays with different number of dimensions

Question

Let's say I have a 2D Numpy array:

>>> a = np.random.random((4,6))

and I want to add a 1D array to each row:

>>> c = np.random.random((6,))
>>> a + c

This works. Now if I try adding a 1D array to each column, I get an error:

>>> b = np.random.random((4,))
>>> a + b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape

I can fix this by using np.newaxis:

>>> a + b[:,np.newaxis]

which does work as expected.

What are the shape-matching rules to avoid having to use np.newaxis? Is it that the last element of the numpy shape tuple has to match? Does this rule also apply to higher dimensions? For example, the following works:

>>> a = np.random.random((2,3,4,5))
>>> b = np.random.random((4,5))
>>> a + b

So my question is whether this is documented anywhere, and if it is a behavior that can be relied on, or whether it is best to always use np.newaxis?

Answer 1

This is a distinctive feature of numpy called 'broadcasting': if you can multiply a vector by a scalar why not allow multiplying a matrix by a vector? Just like every element of a vector is multiplied by a scalar in the first case, every cell in the matrix row is multiplied by the corresponding vector element in the second case.

Broadcasting is done using four rules which are a bit complicated in formulation but are rather intuitive once understood:

1. All input arrays with ndim smaller than the input array of largest ndim, have 1’s prepended to their shapes.
2. The size in each dimension of the output shape is the maximum of all the input sizes in that dimension.
3. An input can be used in the calculation if its size in a particular dimension either matches the output size in that dimension, or has value exactly 1.
4. If an input has a dimension size of 1 in its shape, the first data entry in that dimension will be used for all calculations along that dimension. In other words, the stepping machinery of the ufunc will simply not step along that dimension (the stride will be 0 for that dimension).

For example,

The operation is possible (doesn't result in shape mismatch error you mentioned) in three cases:

1. The arrays all have exactly the same shape.
2. The arrays all have the same number of dimensions and the length of each dimensions is either a common length or 1.
3. The arrays that have too few dimensions can have their shapes prepended with a dimension of length 1 to satisfy property 2.

More examples can be found in my recent article on broadcasting [1] or in the official docs [2].

References

1. Broadcasting in NumPy, 2021 https://towardsdatascience.com/58856f926d73
2. NumPy Reference. Broadcasting http://docs.scipy.org/doc/numpy/reference/ufuncs.html#broadcasting