Synchronizing images from two different cameras

Question

I am receiving images from two different cameras, having different frame rates and resolutions. One of the cameras (the smart camera) works as a controller. It receives images from other camera and its own image sensor and then saves the image pairs on the on-board memory. Here, I have a problem. I have to do stereo matching with each pair of images. Hence, the images should be perfectly synchronized (just a few milliseconds apart). I point both the cameras to an on-screen stop watch and capture images just to see how far, in time, they are from each other. It turned out to be disappointing that neither they are synchronized, neither the time offset between a single pair is constant. I am working on linux and programming in C. I tried two posix threads for capturing images from individual cameras. The first thread cam1() signals the other thread cam2() before it starts image capture. Therefore, the two image captures are supposed to start at the same time. But this doesn't work either. I would really appreciate any solution in this regard.

Answer 1

Clock synchronization is a pretty challenging issue in every sensor network. I am not an expert, however there are some solutions you can try.

Firstly, n.m. is right! software synchronization is very hard in high FPS situations where you need to control the FPS (you need to have the same FPS), the exposure, timestamps etc between the image acquiring of the two cameras. Camera synchronization is commonly done externally known as external trigger and sync I/O.

In externally synchronized cameras the two clocks aren't always the same. The camera clock crystal's always have a drift which is specified by design. you can calculate how much time it takes for two images to drift in a distinguishable manner and reset the clocks. I have used AVT cameras for some time. The company proposes:

Users interested in synching the camera Timestamp to another clock should note that the camera clock will drift from the other clock, and therefore will need to be periodically re-synched by resetting the Timestamp.

If you have low FPS and you can know the dropped frames count at any time, you can synchronize with the (increasing) annotation named Frame Count of each frame.

If you have stereo calibrated images, certain features in the default camera's images (define one arbitrarily or just the one that triggers the other), may lay on/close to the epipolar lines that cross the other camera. Thus epipolar lines may introduce a certain invariant in your program logic to pair features (detected blobs).