Do numpy 1D arrays follow row/column rules?

Question

I have just started using numpy and I am getting confused about how to use arrays. I have seen several Stack Overflow answers on numpy arrays but they all deal with how to get the desired result (I know how to do this, I just don't know why I need to do it this way). The consensus that I've seen is that arrays are better than matrices because they are a more basic class and less restrictive. I understand you can transpose an array which to me means there is a distinction between a row and a column, but the multiplication rules all produce the wrong outputs (compared to what I am expecting).

Here is the test code I have written along with the outputs:

a = numpy.array([1,2,3,4])
print(a)
>>> [1 2 3 4]

print(a.T)          # Transpose
>>> [1 2 3 4]       # No apparent affect

b = numpy.array( [ [1], [2], [3], [4] ] )
print(b)
>>> [[1]
     [2]
     [3]
     [4]]           # Column (Expected)

print(b.T)
>>> [[1 2 3 4]]     # Row (Expected, transpose seems to work here)

print((b.T).T)
>>> [[1]
     [2]
     [3]
     [4]]           # Column (All of these are as expected, 
                    #          unlike for declaring the array as a row vector)

# The following are element wise multiplications of a
print(a*a)
>>> [ 1  4  9 16]

print(a * a.T)      # Row*Column
>>> [ 1  4  9 16]   # Inner product scalar result expected

print(a.T * a)      # Column*Row
>>> [ 1  4  9 16]   # Outer product matrix result expected

print(b*b)
>>> [[1]
     [4]
     [9]
     [16]]          # Expected result, element wise multiplication in a column

print(b * b.T)      # Column * Row (Outer product)
>>> [[ 1  2  3  4]
     [ 2  4  6  8]
     [ 3  6  9 12]
     [ 4  8 12 16]] # Expected matrix result

print(b.T * (b.T))  # Column * Column (Doesn't make much sense so I expected elementwise multiplication
>>> [[ 1  4  9 16]]

print(b.T * (b.T).T) # Row * Column, inner product expected
>>> [[ 1  2  3  4]
    [ 2  4  6  8]
    [ 3  6  9 12]
    [ 4  8 12 16]]  # Outer product result

I know that I can use numpy.inner() and numpy.outer() to achieve the affect (that is not a problem), I just want to know if I need to keep track of whether my vectors are rows or columns.

I also know that I can create a 1D matrix to represent my vectors and the multiplication works as expected. I'm trying to work out the best way to store my data so that when I look at my code it is clear what is going to happen - right now the maths just looks confusing and wrong.

I only need to use 1D and 2D tensors for my application.

Answer 1

I'll try annotating your code

a = numpy.array([1,2,3,4])
print(a)
>>> [1 2 3 4]

print(a.T)          # Transpose
>>> [1 2 3 4]       # No apparent affect

a.shape will show (4,). a.T.shape is the same. It kept the same number of dimensions, and performed the only meaningful transpose - no change. Making it (4,1) would have added a dimension, and destroyed the A.T.T roundtrip.

b = numpy.array( [ [1], [2], [3], [4] ] )
print(b)
>>> [[1]
     [2]
     [3]
     [4]]           # Column (Expected)

print(b.T)
>>> [[1 2 3 4]]     # Row (Expected, transpose seems to work here)

b.shape is (4,1), b.T.shape is (1,4). Note the extra set of []. If you'd created a as a = numpy.array([[1,2,3,4]]) its shape too would have been (1,4).

The easy way to make b would be b=np.array([[1,2,3,4]]).T (or b=np.array([1,2,3,4])[:,None] or b=np.array([1,2,3,4]).reshape(-1,1))

Compare this to MATLAB

octave:3> a=[1,2,3,4]
a =
   1   2   3   4
octave:4> size(a)
ans =
   1   4
octave:5> size(a.')
ans =
   4   1

Even without the extra [] it has initialed the matrix as 2d.

numpy has a matrix class that imitates MATLAB - back in the time when MATLAB allowed only 2d.

In [75]: m=np.matrix('1 2 3 4')

In [76]: m Out[76]: matrix([[1, 2, 3, 4]])

In [77]: m.shape
Out[77]: (1, 4)

In [78]: m=np.matrix('1 2; 3 4')

In [79]: m
Out[79]: 
matrix([[1, 2],
        [3, 4]])

I don't recommend using np.matrix unless it really adds something useful to your code.

Note the MATLAB talks of vectors, but they are really just their matrix with only one non-unitary dimension.

# The following are element wise multiplications of a
print(a*a)
>>> [ 1  4  9 16]

print(a * a.T)      # Row*Column
>>> [ 1  4  9 16]   # Inner product scalar result expected

This behavior follows from a.T == A. As you noted, * produces element by element multiplication. This is equivalent to the MATLAB .*. np.dot(a,a) gives the dot or matrix product of 2 arrays.

print(a.T * a)      # Column*Row
>>> [ 1  4  9 16]   # Outer product matrix result expected

No, it is still doing elementwise multiplication.

I'd use broadcasting, a[:,None]*a[None,:] to get the outer product. Octave added this in imitation of numpy; I don't know if MATLAB has it yet.

In the following * is always element by element multiplication. It's broadcasting that produces matrix/outer product results.

print(b*b)
>>> [[1]
     [4]
     [9]
     [16]]          # Expected result, element wise multiplication in a column

A (4,1) * (4,1)=>(4,1). Same shapes all around.

print(b * b.T)      # Column * Row (Outer product)
>>> [[ 1  2  3  4]
     [ 2  4  6  8]
     [ 3  6  9 12]
     [ 4  8 12 16]] # Expected matrix result

Here (4,1)*(1,4)=>(4,4) product. The 2 size 1 dimensions have been replicated so it becomes, effectively a (4,4)*(4,4). How would you do replicate this in MATLAB - with .*?

print(b.T * (b.T))  # Column * Column (Doesn't make much sense so I expected elementwise multiplication
>>> [[ 1  4  9 16]]

* is elementwise regardless of expectations. Think b' .* b' in MATLAB.

print(b.T * (b.T).T) # Row * Column, inner product expected
>>> [[ 1  2  3  4]
    [ 2  4  6  8]
    [ 3  6  9 12]
    [ 4  8 12 16]]  # Outer product result

Again * is elementwise; inner requires a summation in addition to multiplication. Here broadcasting again applies (1,4)*(4,1)=>(4,4).

np.dot(b,b) or np.trace(b.T*b) or np.sum(b*b) give 30.

When I worked in MATLAB I frequently checked the size, and created test matrices that would catch dimension mismatches (e.g. a 2x3 instead of a 2x2 matrix). I continue to do that in numpy.

The key things are:

• numpy arrays may be 1d (or even 0d)

• A (4,) array is not exactly the same as a (4,1) or (1,4)`.

• * is elementwise - always.

• broadcasting usually accounts for outer like behavior