Align two images in OpenCV

Question

I have two images (see below). These images represent the contours of a pair of cables and were captured using laser based 3D triangulation. The first image is captured with the left camera, while the second one with the right camera. As can be seen, these images are partially overlapping. The left part on the first image partly corresponds to the left part on the second image. The same holds for the right part. I want to merge these two images into one image so that the corresponding parts overlap.

Next to these images, I also have the following information at my disposal:

• A 3x3 homography matrix H of left and right camera
• Intrinsic camera parameters K of left and right camera
• Distortion coefficients D (9 of them) of left and right camera
• Offset O of left and right camera

This data is specified below.

In Halcon, I have tried to do this using mosaicking:

• Extract characteristic points in both images using Harris
• Compute a projective transformation matrix from one image to the other using Ransac.
• Apply the found projective transformation matrix.

This approach was however not successful. I am looking for a similar approach in OpenCV or Halcon or an approach (also in OpenCV or Halcon) that makes use of the calibration data I have at my disposal, such as the homography matrix and camera matrix.

Please provide ample explanations, if possible, since I am only starting out with Machine Vision.

Hl := [0.00175186,   4.73083e-05, -0.00108921,
       0.000780817, -0.00145615,   0.00118631,
       0.0534139,   -0.030823,     1.0        ]
Kl := [4578.21,   -5.05144, 759.766,
       0.0,     4576.87,    568.223,
       0.0,        0.0,       1.0   ]
Dl := [-0.12573, 0.0533453, -0.575361, -0.0130272, 0.00348033, 0.00852617, -0.0271142, 0.0176706, -0.00575124]
Ol := [0.0, 150.0]

Hr := [0.00173883, -2.94597e-05, 0.00109873,
      -0.00077676, -0.0014687,   0.00121393,
      -0.0653829,  -0.0443924,   1.0        ]
Kr := [4591.96,  -4.55317, 1284.74,
       0.0,    4591.19,     534.317,
       0.0,       0.0,        1.0   ]
Dr := [-0.110751, -0.349716, 3.86535, 0.017393, -0.00364957, -0.00633656, 0.0338833, -0.0212222, 0.00543694]
Or := [0.0, 100.0]

Answer 1

Template matching would do the trick here. I played with it a little, hope you find it usuful (Code below):

MAX_DISPARITY = 100;
imgL=double(imread('https://i.stack.imgur.com/y5tOJ.png'));
imgR=double(imread('https://i.stack.imgur.com/L1EQy.png'));
imgRfused = imgR;
minmax = @(v) [min(v) max(v)];
[imgLbw,n]=bwlabel(imgL);
nBlobs=2;
a=arrayfun(@(i) sum(imgLbw(:)==i),1:n);
[~,indx]=sort(a,'descend');
imgLbwC=bsxfun(@eq,imgLbw,permute(indx(1:nBlobs),[3 1 2]));
imgLbwC =bsxfun(@times,imgLbwC,2.^permute(0:nBlobs-1,[3 1 2]));
imgLbwC  = sum(imgLbwC ,3);

src = zeros(nBlobs,4);
dst = zeros(nBlobs,4);

for i=1:nBlobs
    [y,x]=find(imgLbwC==i);
    mmx = minmax(x);
    mmy = minmax(y);
    ker = imgL(mmy(1):mmy(2),mmx(1):mmx(2));

    [yg,xg]=ndgrid(mmy(1):mmy(2),mmx(1):mmx(2));
    src(i,:)=[mmx(1) mmy(1) fliplr(size(ker))];


    imgR_ = imgR(:,mmx(1)-MAX_DISPARITY:mmx(2)+MAX_DISPARITY);
    c=conv2(imgR_ ,rot90(double(ker),2),'valid')./sqrt(conv2(imgR_.^2,ones(size(ker)),'valid'));
    [yy,xx]=find(c==max(c(:)),1);
    dst(i,:)=[src(i,1:2)+[xx yy-mmy(1)]+[-MAX_DISPARITY,0] fliplr(size(ker))];

    imgRfused(dst(i,2):dst(i,2)+dst(i,4),dst(i,1):dst(i,1)+dst(i,3)) = max(imgRfused(dst(i,2):dst(i,2)+dst(i,4),dst(i,1):dst(i,1)+dst(i,3)),imgL(src(i,2):src(i,2)+src(i,4),src(i,1):src(i,1)+src(i,3)));
end
imagesc(imgRfused);
axis image
colormap gray