opencv single h264 raw frame as a binary string

Question

have created a rtsp client in python that receives a h264 stream and returns single h264 raw frames as a binary strings. I am trying to process each h264 frames on-the-fly.

I have unsuccessfully tried several ways to convert this frame into a numpy array for processing.

So far I know that cv2.VideoCapture only accepts a file name as it argument, not a frame neither a StringIO object (file like pointer to a buffer), but I need to pass to it my string.

I have also tried something like:

nparr = np.fromstring(frame_bin_str, np.uint8)
img_np = cv2.imdecode(nparr, cv2.CV_LOAD_IMAGE_COLOR)

tried diferent flags. but also failed miserably.

after many other failed attempts , I ran out of ideas.

To summarize what I need to do: I have a h264 raw frame in a variable and I need to create an openvc valid numpy array of it, or somehow end up with a VideoCapture object containing that single frame, so I can process the frame.

Any pointers would be much appreciated.

Hope this all makes sense.

Thank you in advance

Answer 1

As Micka suggested, there is no support for h264 RAW format in OpenCV and we should convert it ourselves.

I think you should be reshaping the nparr to the shape of the incoming image. Not necessary to do imdecode. Use imshow to display the result and verify.

Here is the code I used to convert a 16 bit RAW image (grayscale) in a similar way. I have renormalized my image before displaying.

framenp = np.fromstring(framestr, dtype=np.uint16).reshape((1024,1280))
#renormalizing to float
framenp = (framenp*1./framenp.max())
framenp.dtype = np.float
cv2.imshow('frame', cv2.resize(framenp, (640,480)))