How can you visualize the sparsity pattern of a large sparse matrix?
The matrix is too large to fit in memory as a dense array, so I have it in csr_matrix format. When I try pylab's matshow with it, I get the following error:
ValueError: need more than 0 values to unpack
Thoughts?
e.g.:
import pylab as pl
import scipy.sparse as sp
from random import randint
mat = sp.lil_matrix( (4000,3000), dtype='uint8' )
for i in range(1000):
mat[randint(0,4000),randint(0,3000)] = randint(0,10)
pl.figure()
pl.matshow(mat)
One way to visualize sparse matrix is to use 2d plot. Python's matplotlib has a special function called Spy for visualizing sparse matrix. Spy is very similar to matplotlib's imshow, which is great for plotting a matrix or an array as an image. imshow works with dense matrix, while Spy works with sparse matrix.
The solution to representing and working with sparse matrices is to use an alternate data structure to represent the sparse data. The zero values can be ignored and only the data or non-zero values in the sparse matrix need to be stored or acted upon.
Using sparse matrices to store data that contains a large number of zero-valued elements can both save a significant amount of memory and speed up the processing of that data. sparse is an attribute that you can assign to any two-dimensional MATLAB® matrix that is composed of double or logical elements.
matshow
works on dense arrays. For sparse arrays you can use spy
:
import scipy.sparse as sps
import matplotlib.pyplot as plt
a = sps.rand(1000, 1000, density=0.001, format='csr')
plt.spy(a)
plt.show()
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With