Why does summing a sparse matrix leave an empty dimension?

Question

When summing a numpy.ndarray, it reduces the dimensionality of the array, unless we set keepdims=True. However, this doesn't seem to be true for Scipy's sparse matrices:

import scipy.sparse

matrix = scipy.sparse.coo_matrix([[0, 1], [2, 1]])
print(matrix.shape)                 # (2, 2) as expected.
print(matrix.sum().shape)           # () as expected.
print(matrix.sum(axis=0).shape)     # (1, 2) but expected (2,).
print(matrix.sum(axis=0)[0].shape)  # (1, 2) but expected (2,).

As shown from the last line in the example, I cannot even select the resulting vector. Further, trying to cast the result of the sum to a dense Numpy array fails:

matrix.toarray()              # This works.
matrix.sum(axis=0).toarray()  # AttributeError: 'matrix' has no 'toarray'.

How can I compute the sum of a sparse matrix along one dimension and obtain the result as a dense array?

Answer 1

A sparse matrix, while a distinct class (classes depending on format), tries to behave like np.matrix (which in turn behaves like an old fashioned MATLAB matrix). Its shape is always 2d, and indexing, summing, and related actions, returns 2d.

In [172]: M = sparse.csr_matrix([[0,1],[2,1]])
In [173]: M
Out[173]: 
<2x2 sparse matrix of type '<class 'numpy.int32'>'
    with 3 stored elements in Compressed Sparse Row format>

Total sum produces a scalar, or 0d array:

In [174]: M.sum()
Out[174]: 4
In [175]: _.shape
Out[175]: ()
In [176]: type(__)
Out[176]: numpy.int32

Axis sum produces a 2d column or row vector dense matrix:

In [177]: M.sum(axis=0)
Out[177]: matrix([[2, 2]], dtype=int32)
In [178]: _.shape
Out[178]: (1, 2)
In [179]: M.sum(axis=1)
Out[179]: 
matrix([[1],
        [3]])
In [180]: _.shape
Out[180]: (2, 1)

This is the same behavior you get from a dense matrix, e.g. M.todense(). keepdims=True provides something like this for arrays (people coming from MATLAB complain about the sum reducing the dimensions).

A dense matrix has a handy .A1 property that converts it to a 1d array:

In [181]: M.sum(axis=1).A1
Out[181]: array([1, 3])
In [182]: M.sum(axis=0).A1
Out[182]: array([2, 2], dtype=int32)

.A works for both sparse matrix and dense matrix, but only sparse has a toarray method (and todense). Like I said, sparse imitates the dense matrix, but isn't a subclass.

Sparse axis sum actually does a matrix multiplication; dense matrix * sparse matrix produces a dense matrix:

In [186]: M*np.matrix([[1],[1]])
Out[186]: 
matrix([[1],
        [3]], dtype=int32)
In [187]: np.matrix([[1,1]])*M
Out[187]: matrix([[2, 2]], dtype=int32)

A complaint about array sum not leaving this 'empty' (singular is a term, 1 not 0) dimension:

Why does the shape remains same when I sum a square numpy array along either directions?