"Too many values to unpack" in numpy histogram2d

Question

I am using numpy histogram2d to compute the values for the visual representation of a 2d histogram of two variables:

H, xedges, yedges = np.histogram2d(Z[:,0], Z[:,1], bins=100)

where Z is a numpy matrix

The error that I'm getting is:

Traceback (most recent call last):
File "/home/.../pca_analysis.py", line 141, in <module>
   H, xedges, yedges = np.histogram2d(Z[:,0], Z[:,1], bins=100)
File "/usr/lib/python2.7/dist-packages/numpy/lib/twodim_base.py", line 615, in histogram2d
   hist, edges = histogramdd([x,y], bins, range, normed, weights)
File "/usr/lib/python2.7/dist-packages/numpy/lib/function_base.py", line 281, in histogramdd
   N, D = sample.shape
ValueError: too many values to unpack

I cannot really understand why I am getting this error. I have tried using the histogram2d function with random values and it is working properly. I have also tried to transform both Z[:,0] and Z[:,1] in numpy arrays and simple lists, but I'm getting the same problem.

Answer 1

As @seberg notes in the comments, Z is a matrix, so it must be cast as an array before slicing.

np.asarray(Z)[:,0]

The reason this is necessary is because the np.matrix maintains its two-dimensionality even after slicing, so that the column of matrix has shape (N,1), not (N,) as the histogram functions expect.

The reason it doesn't work to cast to an array after slicing is that the shape is unchanged by casting; the behavior of slicing is what is different.

In case that doesn't make sense, here's an illustration:

In [4]: a = np.arange(9).reshape(3,3)

In [5]: a
Out[5]: 
array([[0, 1, 2],
       [3, 4, 5],
       [6, 7, 8]])

In [6]: m = np.matrix(a)

In [7]: m
Out[7]: 
matrix([[0, 1, 2],
        [3, 4, 5],
        [6, 7, 8]])

In [8]: m[:,0]
Out[8]: 
matrix([[0],
        [3],
        [6]])

In [9]: a[:,0]
Out[9]: array([0, 3, 6])

In [10]: m[:,0].shape
Out[10]: (3, 1)

In [11]: a[:,0].shape
Out[11]: (3,)

If you cast after slicing, the shape is still 2d:

In [12]: np.array(m[:,0])
Out[12]: 
array([[0],
       [3],
       [6]])

In [13]: np.array(m[:,0]).shape
Out[13]: (3, 1)