Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

matshow with sparse matrices

How can you visualize the sparsity pattern of a large sparse matrix?

The matrix is too large to fit in memory as a dense array, so I have it in csr_matrix format. When I try pylab's matshow with it, I get the following error:

ValueError: need more than 0 values to unpack

Thoughts?

e.g.:

import pylab as pl
import scipy.sparse as sp
from random import randint

mat = sp.lil_matrix( (4000,3000), dtype='uint8' )
for i in range(1000):
    mat[randint(0,4000),randint(0,3000)] = randint(0,10)

pl.figure()
pl.matshow(mat)
like image 886
Alexander Tronchin-James Avatar asked Sep 11 '13 18:09

Alexander Tronchin-James


People also ask

How do you visualize a sparse matrix?

One way to visualize sparse matrix is to use 2d plot. Python's matplotlib has a special function called Spy for visualizing sparse matrix. Spy is very similar to matplotlib's imshow, which is great for plotting a matrix or an array as an image. imshow works with dense matrix, while Spy works with sparse matrix.

How does machine learning deal with sparse matrix?

The solution to representing and working with sparse matrices is to use an alternate data structure to represent the sparse data. The zero values can be ignored and only the data or non-zero values in the sparse matrix need to be stored or acted upon.

What are sparse matrix used for?

Using sparse matrices to store data that contains a large number of zero-valued elements can both save a significant amount of memory and speed up the processing of that data. sparse is an attribute that you can assign to any two-dimensional MATLAB® matrix that is composed of double or logical elements.


1 Answers

matshow works on dense arrays. For sparse arrays you can use spy:

import scipy.sparse as sps
import matplotlib.pyplot as plt

a = sps.rand(1000, 1000, density=0.001, format='csr')

plt.spy(a)
plt.show()

enter image description here

like image 167
Jaime Avatar answered Oct 17 '22 17:10

Jaime