Adding matrices with different dimensions

Question

In numpy and tensorflow it's possible to add matrices (or tensors) of different dimensionality if the shape of smaller matrix is a suffix of bigger matrix. This is an example:

x = np.ndarray(shape=(10, 7, 5), dtype = float)
y = np.ndarray(shape=(7, 5), dtype = float)

For these two matrices operation x+y is a shortcut for:

for a in range(10):
    for b in range(7):
        for b in range(5):
            result[a,b,c] = x[a,b,c] + y[b,c]

In my case however I have matrices of shape (10,7,5) and (10,5) and likewise I would like to perform the + operation using similar logic:

for a in range(10):
    for b in range(7):
        for b in range(5):
            result[a,b,c] = x[a,b,c] + y[a,c]
                                         ^

In this case however x+y operation fails as neither numpy nor tensorflow understands what I want to do. Is there any way I can perform this operation efficiently (without writing python loop myself)?

In so far I have figured that I could introduce a temporary matrix z of shape (10,7,5) using einsum like this:

z = np.einsum('ij,k->ikj', y, np.ones(7))
x + z

but this creates an explicit three-dimensional matrix (or tensor) and if possible I would prefer to avoid that.

Answer 1

In NumPy, you could extend y to 3D and then add -

x + y[:,None,:]

Haven't dealt with tensorflow really, but looking into its docs, it seems, we could use tf.expand_dims -

x + tf.expand_dims(y, 1)

The extended version would still be a view into y and as such won't occupy anymore of the memory, as tested below -

In [512]: np.may_share_memory(y, y[:,None,:])
Out[512]: True

Answer 2

As correctly pointed out in the accepted answer the solution is to expand dimensions using available construct.

The point is to understand how numpy is doing the broadcasting of matrices in case of adding matrices if their dimensions don't match. The rule is that two matrices must have exactly the same dimensions with an exception that some dimensions in either of matrices can be replaced by 1.

E.g.

A      (4d array):  8 x 1 x 6 x 1
B      (3d array):      7 x 1 x 5
Result (4d array):  8 x 7 x 6 x 5

This example and detailed explanation can be found in scipy docs https://docs.scipy.org/doc/numpy-1.10.0/user/basics.broadcasting.html