Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

OpenCV Canny + Watershed

I'm using a canny edge detection and a finding contours function (both OpenCV) to create markers for the watershed transform. Everything works fine but I'm not 100% satisfied with the results. The reason is that some edges are missing and therefore important information is lost. In more detail, I got a bunch of windows (front views), which are rectangles, after the watershed transform I end up with something like this:

enter image description hereenter image description here but I would rather have nice rectangles, that are complete and not open to one side. While maintaining irregular shapes (bushes in front of the house, cars..) Any ideas how I could solve this problem?I thought about overlaying the whole image with a grid, but I can't make it work.

Thank you very much.

Here is my code:

Mat gray;
cvtColor(im, gray, CV_BGR2GRAY);

// Use Canny instead of threshold to catch squares with gradient shading
Mat bw;
Canny(gray, bw, 0, 100, 5, true);

// Find contours
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;

findContours( bw, contours, hierarchy,
    CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );


// watershed
Mat markers(bw.size(), CV_32S);
markers = Scalar::all(0);
int idx = 0;
int compCount = 0;
for( ; idx >= 0; idx = hierarchy[idx][0], compCount++ ) {
    if (fabs(contourArea(contours[compCount])) < min_size )
        continue;
    drawContours(markers, contours, idx, Scalar::all(compCount+1), 1, 8, hierarchy, INT_MAX);
}
watershed( im, markers );

As requested, here is the original image, the image I would like to get and my output: enter image description here

And I would like to have a segmentation like this (although over segmentation does not hurt, I just need to make sure, I get all the details):

enter image description here

While I get something like this: enter image description here (please ignore the colours, they are not important for this question and are just a result of my overall program). This is only one example, if you want, I can show you more, also please have a look at the etrims dataset, all my pictures are from there.

like image 736
user667804 Avatar asked Jan 11 '14 09:01

user667804


1 Answers

Two things -

1) As already mentioned, edge detection results in spurious edges being picked up.

2) Using these edges as markers for watershed segmentation results in over-segmentation because every marker produces a segmented region in the output.

Strategy -

(i) Preprocessing: Smooth the image heavily (morphological opening by reconstruction can be used for homogenizing the intensities without significantly affecting edges you are interested in).

(ii) Markers: Instead of using edges as seeds, I'd use the local extrema. Ideally, we want one marker for every region we want segmented.

(iii) Segmentation: Find the gradient magnitude (range filtering is also a good option) of the image from step (i) and use that as the segmentation function.

Using this strategy, I get the following segmentation.

enter image description here

Alternatively, after step (i), you can use Canny edge detection and do some morphological cleanup (to fill contours and remove edges that remain). This is what I get.

enter image description here

These are not exactly the expected segmentation (some objects like the car are not detected), but are a good start.

Edit: The MATLAB code used to generate the images -

% convert to grayscale
img = rgb2gray(origImg);

% create an appropriate structuring element
w_size = 20;
seSquare = strel('square', w_size);

% opening by reconstruction - to smooth dark regions
imgEroded = imerode(img, seSquare);
imgRecon = imreconstruct(imgEroded, img);

% invert and repeat - to smooth bright regions
imgReconComp = imcomplement(imgRecon);
imgEroded2 = imerode(imgReconComp, seSquare);
imgRecon2 = imreconstruct(imgEroded2, imgReconComp);

% get foreground markers
fgm = imregionalmax(imgRecon2);

% get background markers - this step can be skipped 
% in which case only fgm would be the marker image 
% and the segmentation would be different 
distTrans = bwdist(fgm);
wLines= watershed(distTrans);
bgm = wLines == 0;

% get the segmentation function and impose markers
% perform watershed segmentation
seSquare3 = strel('square', 3);
rangeImg = rangefilt(imgRecon2, getnhood(seSquare3));
segFunc = imimposemin(rangeImg, fgm | bgm);
grayLabel = watershed(segFunc);

rgbLabel= label2rgb(grayLabel);
figure, imshow(rgbLabel); title('Output using Watershed')

% alternatively, extract edges from the preprocessed image
% perform morph cleanup
bwEdges = edge(imgRecon2, 'canny');
bwFilled = imfill(bwEdges, 'holes');
bwRegions = imopen(bwFilled, seSquare3);
grayLabel = bwlabel(bwRegions);

rgbLabel = label2rgb(grayLabel, 'jet', 'k');
figure, imshow(rgbLabel); title('Output using Canny')
like image 181
zepman Avatar answered Oct 08 '22 16:10

zepman