Display 2D array from OpenCV in Matplotlib.pyplot.imshow()

Question

I have a setup in Python where I use the OpenCV library to read the frames of a video in as 2D arrays--these videos are grayscale, so I'm using unsigned 8-bit integers for the pixel values.

In my next step, I'm trying to use pyplot.imshow() from matplotlib to display the frame of the video. Unfortunately, I'm getting something that doesn't make any sense at all.

Here's the code to read the video:

import numpy as np
import cv

def read_video(filename):
  video = cv.CaptureFromFile('%s' % filename)
  num_frames = int(cv.GetCaptureProperty(video, cv.CV_CAP_PROP_FRAME_COUNT))

  frames = []
  for i in range(0, num_frames):
    frame = cv.QueryFrame(video)
    if frame is None:
      quit('Failed to extract frame %s of %s!' % (i, num_frames))
    toadd = cv2numpy(frame, 'uint8')
    frames.append(np.array(toadd))
  return np.array(frames)

cv2numpy is a utility function that converts the OpenCV array to a numpy array (just a call to fromstring and then a reshape). Here's the code I'm using to plot the first frame of the video:

import matplotlib.pyplot as plot
import matplotlib.cm as cm

frames = read_video('video.avi')
plot.imshow(frames[0], cmap = cm.gray)
plot.show()

In some other code, I used the OpenCV SaveImage on a single frame to provide a reference for what I would expect from imshow. Here's the image I get from the former, and here's the image I get from the code above.

As you can see they're wildly different. The only thing I can glean from the actual image is the striping: it looks almost like it's getting the dimensions wrong, that there are more pixels in the width than the height (this image is supposed to be 128 x 256). But I've tried transposing the array before plotting it, changing the extent and aspect and shape parameters as per the imshow documentation, and with the exception of some bizarre pixel stretching I haven't found a fix.

Any thoughts?

EDIT 1: I figure it may be prudent to add the cv2numpy code, in case that reshaping is somehow muddling things (since my "truth" image above does not use that code and hence cv2numpy is involved only in the questionable pipeline).

def cv2numpy(cvarr, the_type):
  a = np.fromstring(
      cvarr.tostring(),
      dtype = the_type,
      count = cvarr.width * cvarr.height)
  a.shape = (cvarr.height, cvarr.width)
  return a

Answer 1

I believe that the problem is with your cv2numpy function. Try this one:

def cv2numpy(cvarr, the_type):
  a = np.asarray(cv.GetMat(cvarr), dtype=the_type)
  return a

It did the trick for me. If you aren't using a grayscale input (I know that you said you are using grayscale now) then you will need to convert using cv.CreateImage and cv.CvtColor.

Answer 2

Are you using version 2.3.1? Using cv2 API, we don't need implement our own version of OpenCV/Numpy conversion anymore. For example, the following code works just right:

>>> import cv2
>>> from matplotlib import pyplot as plt
>>> lenna = cv2.imread('lenna.tiff', cv2.CV_LOAD_IMAGE_GRAYSCALE)
>>> lenna
array([[162, 162, 162, ..., 170, 155, 128],
       [162, 162, 162, ..., 170, 155, 128],
       [162, 162, 162, ..., 170, 155, 128],
       ..., 
       [ 43,  43,  50, ..., 104, 100,  98],
       [ 44,  44,  55, ..., 104, 105, 108],
       [ 44,  44,  55, ..., 104, 105, 108]], dtype=uint8)
>>> plt.imshow(lenna, cmap='gray')
>>> plt.show()