NumPy - What is broadcasting?

Question

I don't understand broadcasting. The documentation explains the rules of broadcasting but doesn't seem to define it in English. My guess is that broadcasting is when NumPy fills a smaller dimensional array with dummy data in order to perform an operation. But this doesn't work:

>>> x = np.array([1,3,5])
>>> y = np.array([2,4])
>>> x+y
*** ValueError: operands could not be broadcast together with shapes (3,) (2,) 

The error message hints that I'm on the right track, though. Can someone define broadcasting and then provide some simple examples of when it works and when it doesn't?

Answer 1

The term broadcasting describes how numpy treats arrays with different shapes during arithmetic operations.

It's basically a way numpy can expand the domain of operations over arrays.

The only requirement for broadcasting is a way aligning array dimensions such that either:

• Aligned dimensions are equal.
• One of the aligned dimensions is 1.

So, for example if:

x = np.ndarray(shape=(4,1,3))
y = np.ndarray(shape=(3,3))

You could not align x and y like so:

4 x 1 x 3
3 x 3

But you could like so:

4 x 1 x 3
    3 x 3

How would an operation like this result?

Suppose we have:

x = np.ndarray(shape=(1,3), buffer=np.array([1,2,3]),dtype='int')
array([[1, 2, 3]])

y = np.ndarray(shape=(3,3), buffer=np.array([1,1,1,1,1,1,1,1,1]),dtype='int')
array([[1, 1, 1],
       [1, 1, 1],
       [1, 1, 1]])

The operation x + y would result in:

array([[2, 3, 4],
       [2, 3, 4],
       [2, 3, 4]])

I hope you caught the drift. If you did not, you can always check the official documentation here.

Cheers!

Answer 2

Broadcasting is numpy trying to be smart when you tell it to perform an operation on arrays that aren't the same dimension. For example:

2 + np.array([1,3,5]) == np.array([3, 5, 7])

Here it decided you wanted to apply the operation using the lower dimensional array (0-D) on each item in the higher-dimensional array (1-D).

You can also add a 0-D array (scalar) or 1-D array to a 2-D array. In the first case, you just add the scalar to all items in the 2-D array, as before. In the second case, numpy will add row-wise:

In [34]: np.array([1,2]) + np.array([[3,4],[5,6]])
Out[34]: 
array([[4, 6],
       [6, 8]])

There are ways to tell numpy to apply the operation along a different axis as well. This can be taken even further with applying an operation between a 3-D array and a 1-D, 2-D, or 0-D array.