Compute curvature of a bent pipe using image processing (Hough transform parabola detection)

Question

I'm trying to design a way to detect this pipe's curvature. I tried applying hough transform and found detected line but they don't lie along the surface of pipe so smoothing it out to fit a beizer curve is not working .Please suggest some good way to start for the image like this.[

The image obtained by hough transform to detect lines is as follows [ I'm using standard Matlab code for probabilistic hough transform line detection that generates line segment surrounding the structure. Essentially the shape of pipe resembles a parabola but for hough parabola detection I need to provide eccentricity of the point prior to the detection. Please suggest a good way for finding discrete points along the curvature that can be fitted to a parabola. I have given tag to opencv and ITK so if there is function that can be implemented on this particular picture please suggest the function I will try it out to see the results.

img = imread('test2.jpg');
rawimg = rgb2gray(img);
 [accum, axis_rho, axis_theta, lineprm, lineseg] = Hough_Grd(bwtu, 8, 0.01);
figure(1); imagesc(axis_theta*(180/pi), axis_rho, accum); axis xy;
 xlabel('Theta (degree)'); ylabel('Pho (pixels)');
 title('Accumulation Array from Hough Transform');
  figure(2); imagesc(bwtu); colormap('gray'); axis image;
  DrawLines_2Ends(lineseg);
  title('Raw Image with Line Segments Detected');

The edge map of the image is as follows and the result generated after applying Hough transform on edge map is also not good. I was thinking a solution that does general parametric shape detection like this curve can be expressed as a family of parabola and so we do a curve fitting to estimate the coefficients as it bends to analyze it's curvature. I need to design a real time procedure so please suggest anything in this direction.

Answer 1

I suggest the following approach:

First stage: generate a segmentation of the pipe.

1. perform thresholding on the image.
2. find connected components in the thresholded image.
3. search for a connected component which represents the pipe. The connected component which represents the pipe should have an edge map which is divided into top and bottom edges (see attached image). The top and bottom edges should have similar size, and they should have a relatively constant distance from one another. In other words, the variance of their per-pixel distances should be low.

Second stage - extract curve

At this stage, you should extract the points of the curve for performing Beizer fitting. You can either perform this calculation on the top edge, or the bottom edge. another option is to do it on the skeleton of the pipe segmentation.

Results

The pipe segmentation. Top and bottom edges are mark with blue and red correspondingly.

Code

I = mat2gray(imread('ILwH7.jpg'));
im = rgb2gray(I);

%constant values to be used later on
BW_THRESHOLD = 0.64;
MIN_CC_SIZE = 50;
VAR_THRESHOLD = 2;
SIMILAR_SIZE_THRESHOLD = 0.85;

%stage 1 - thresholding & noise cleaning
bwIm = im>BW_THRESHOLD;
bwIm = imfill(bwIm,'holes');
bwIm = imopen(bwIm,strel('disk',1));


CC = bwconncomp(bwIm);
%iterates over the CC list, and searches for the CC which represents the
%pipe
for ii=1:length(CC.PixelIdxList)
    %ignore small CC
    if(length(CC.PixelIdxList{ii})<50)
        continue;
    end
    %extracts CC edges
    ccMask = zeros(size(bwIm));
    ccMask(CC.PixelIdxList{ii}) = 1;
    ccMaskEdges = edge(ccMask);

    %finds connected components in the edges mat(there should be two).
    %these are the top and bottom parts of the pipe.
    CC2 = bwconncomp(ccMaskEdges);
    if length(CC2.PixelIdxList)~=2
        continue;
    end

    %tests that the top and bottom edges has similar sizes
    s1 = length(CC2.PixelIdxList{1});
    s2 = length(CC2.PixelIdxList{2});
    if(min(s1,s2)/max(s1,s2) < SIMILAR_SIZE_THRESHOLD)
        continue;
    end

    %calculate the masks of these two connected compnents
    topEdgeMask = false(size(ccMask));
    topEdgeMask(CC2.PixelIdxList{1}) = true;
    bottomEdgeMask = false(size(ccMask));
    bottomEdgeMask(CC2.PixelIdxList{2}) = true;

    %tests that the variance of the distances between the points is low
    topEdgeDists = bwdist(topEdgeMask);
    bottomEdgeDists = bwdist(bottomEdgeMask);
    var1 = std(topEdgeDists(bottomEdgeMask));
    var2 = std(bottomEdgeDists(topEdgeMask));

    %if the variances are low - we have found the CC of the pipe. break!
    if(var1<VAR_THRESHOLD && var2<VAR_THRESHOLD)
        pipeMask = ccMask;
        break;
    end


end

%performs median filtering on the top and bottom boundaries.
MEDIAN_SIZE =5;
[topCorveY, topCurveX] = find(topEdgeMask);
topCurveX = medfilt1(topCurveX);
topCurveY = medfilt1(topCurveY);
[bottomCorveY, bottomCurveX] = find(bottomEdgeMask);
bottomCurveX = medfilt1(bottomCurveX);
bottomCorveY = medfilt1(bottomCorveY);

%display results
imshow(pipeMask); hold on;
plot(topCurveX,topCorveY,'.-');
plot(bottomCurveX,bottomCorveY,'.-');

Comments

1. In this specific example, acquiring the pipe segmentation by thresholding was relatively easy. In some scenes it may be more complex. in these cases, you may want to use region growing algorithm for generating the pipe segmentation.

2. Detecting the connected component which represents the pipe can be done by using some more hueristics. For example - the local curvature of it's boundaries should be low.

Answer 2

You can find the connected components (CCs) of your inverted edge-map image. Then you can somehow filter those components, say for example, based on their pixel count, using region-properties. Here are the connected components I obtained using the given Octave code. Now you can fit a model to each of these CCs using something like nlinfit or any suitable method.

im = imread('uFBtU.png');
gr = rgb2gray(uint8(im));
er = imerode(gr, ones(3)) < .5;

[lbl, n] = bwlabel(er, 8);
imshow(label2rgb(lbl))