How to calculate Depth Information from USB 3.0 Stereo Camera?

Question

I'm interested in using a stereo camera for calculating depth in video/images. The camera is a USB 3.0 Stereoscopic camera from Leopard Imaging https://www.leopardimaging.com/LI-USB30-V024STEREO.html. I'm using MAC OS X btw.

I was told by their customer support that it's a "UVC" camera. When connected to an apple computer it gives a greenish image.

My end goal is to use OpenCV to grab the left and right frames from both lenses so that I can calculate depth. I'm familiar with OpenCV, but not familiar with working with stereo cameras. Any help would be much appreciated. So far I have been doing this in Python 3:

import numpy as np
import cv2
import sys
from matplotlib import pyplot as plt

import pdb; pdb.set_trace()
print("Camera 1 capture", file=sys.stderr)
cap = cv2.VideoCapture(1)

print("Entering while", file=sys.stderr)
while(True):
    _ = cap.grab()
    retVal, frame = cap.retrieve()

    #gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    cv2.imshow('frame', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This works, but it only gives me a green picture/image with no depth. Any advice on how to get both left and right frames from the camera?

Answer 1

The leopard imaging people sent me a clue but I'm not able to make progress because i guess some missing file in my core file. However, I thought it might help somebody. So I'm posting it as an answer. The message was sent by one of the leopard imaging people I contacted through mail. It goes like this.....

We have a customer who successfully separated the two videos on Linux OS one year ago. I tried to contact him to see if he can share the source code with us. Unfortunately, he already left the former company, but he still found some notes (below). Hope it helps.

The camera combines the image from the two sensors into one 16-bit pixel data (The high 8 bits from one camera, and the low 8 bit from the other camera). To fix this problem in linux opencv, should skip color transform by opencv: at

modules/videoio/src/cap_v4l.cpp static IplImage* icvRetrieveFrameCAM_V4L( CvCaptureCAM_V4L* capture, int)

case V4L2_PIX_FMT_YUYV:
#if 1
    /*
    skip color convert. Just copy image buffer to frame.imageData
    */
    memcpy(capture->frame.imageData, capture->buffers[capture->bufferIndex].start, capture->buffers[capture->bufferIndex].length);
#else
    yuyv_to_rgb24(capture->form.fmt.pix.width, capture->form.fmt.pix.height,(unsigned char*) capture->buffers[capture->bufferIndex].start),
            (unsigned char*)capture->frame.imageData);
#endif

Hope this helps.

DISCLAIMER : I went forward with the C++ version(my project was in C++) given in libuvc used the provided routines to obtain the left and right frames separately.