Skip frames and seek to end of RTSP stream in OpenCV

Question

I capture and process an IP camera RTSP stream in a OpenCV 3.4.2 on Raspberry Pi. Unfortunately the processing takes quite a lot of time, roughly 0.2s per frame, and the stream quickly gets delayed.

I don't mind if I skip some frames so I'm looking for a way to seek to the end of the stream before capturing and processing the next frame.

vcap = cv2.VideoCapture("rtsp://{IPcam}/12")

while(1):
    ret, frame = vcap.read()
    time.sleep(0.2)              # <= Simulate processing time
    cv2.imshow('VIDEO', frame)
    if cv2.waitKey(1) == 27:
        break
    vcap.seek_to_end()           # <== How to do this?

How can I do that vcap.seek_to_end() to catch up with the stream, discard the missed frames, and start processing the most current one?

Thanks!

Answer 1

Try this:

vcap = cv2.VideoCapture("rtspsrc location=rtsp://{IPcam}/12 ! decodebin ! videoconvert ! appsink max-buffers=1 drop=true")

This uses gstreamer to grab your camera feed, and will maintain a buffer of length 1 and drop the oldest as new incoming frames are received. Then, every time you call vcap.read() you should get the latest frame.

You can also try using the OMX decoder on the Raspberry Pi if you notice CPU usage is really high, as this will decode the video (assuming it's h264) on the GPU: ! rtph264depay ! h264parse ! omxh264dec ! appsink max-buffers=1 drop=true

You may need to recompile OpenCV as by default it's compiled with FFMPEG support, not gstreamer. This is fairly simple, just pass -D WITH_GSTREAMER=ON -D WITH_FFMPEG=OFF to the cmake command. Make sure you have the gstreamer development libs installed apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev.

Answer 2

I rewrote threading implementation from Shubham into a class, to be able to stream many cameras:

import threading
from threading import Lock
import cv2

class Camera:
    last_frame = None
    last_ready = None
    lock = Lock()

    def __init__(self, rtsp_link):
        capture = cv2.VideoCapture(rtsp_link)
        thread = threading.Thread(target=self.rtsp_cam_buffer, args=(capture,), name="rtsp_read_thread")
        thread.daemon = True
        thread.start()

    def rtsp_cam_buffer(self, capture):
        while True:
            with self.lock:
                self.last_ready, self.last_frame = capture.read()


    def getFrame(self):
        if (self.last_ready is not None) and (self.last_frame is not None):
            return self.last_frame.copy()
        else:
            return None

Then, it can be used as:

capture = Camera('rtsp://...')

while True:
    frame = capture.getFrame()