Numpy broadcasting sliced arrays and vectors

Question

Given three numpy arrays: one multidimensional array x, one vector y with trailing singleton dimension, and one vector z without trailing singleton dimension,

x = np.zeros((M,N))
y = np.zeros((M,1))
z = np.zeros((M,))

the behaviour of broadcast operations changes depending on vector representation and context:

x[:,0] = y      # error cannot broadcast from shape (M,1) into shape (M)
x[:,0] = z      # OK

x[:,0] += y     # error non-broadcastable output with shape (M) doesn't match 
                # broadcast shape(M,M)
x[:,0] += z     # OK

x - y           # OK
x - z           # error cannot broadcast from shape (M,N) into shape (M)

I realize I can do the following:

x - z[:,None]   # OK

but I don't understand what this explicit notation buys me. It certainly doesn't buy readability. I don't understand why the expression x - y is OK, but x - z is ambiguous.

Why does Numpy treat vectors with or without trailing singleton dimensions differently?

edit: The documentation states that: two dimensions are compatible when they are equal, or one of them is 1, but y and z are both functionally M x 1 vectors, since an M x 0 vector doesn't contain any elements.

Answer 1

The convention is that broadcasting will insert singleton dimensions at the beginning of an array's shape. This makes it convenience to perform operations over the last dimensions of an array, so (x.T - z).T should work.

If it were to automatically decide which axis of x was matched by z, an operation like x - z would result in an error if and only if N == M, making code harder to test. So the convention allows some convenience, while being robust to some error.

If you don't like the z[:, None] shorthand, perhaps you find z[:, np.newaxis] clearer.

For an assignment like x[:,0] = y to work, you can use x[:,0:1] = y instead.