Finding the "fitLine " of contours in OpenCV

Question

I'm a program that find contours in a stream, for example :

I want to find "set of points " that can describe this contours say like the red line :

the yellow part is the the moments of the contours ,

I tried to use fitLine function of opencv but the result was nonsense, any Idea how can I get the middle line of a contours, that should represent the key aspect of my Contours . by the way **I'm not asking for codes ! ** just a hint how can I do that ?

thanks in advance for any help

Answer 1

maybe try this approach with a distance transform and ridge detection:

cv::Mat input = cv::imread("fitLine.jpg");
cv::Mat gray;
cv::cvtColor(input,gray,CV_BGR2GRAY);

cv::Mat mask = gray>100;
cv::imshow("mask",mask);

cv::Mat dt;
cv::distanceTransform(mask,dt,CV_DIST_L1,CV_DIST_MASK_PRECISE);

cv::imshow("dt", dt/15.0f);
cv::imwrite("fitLineOut.png",255*dt/15.0f);


//care: this part doesn't work for diagonal lines, a ridge detection would be better!!
cv::Mat lines = cv::Mat::zeros(input.rows, input.cols, CV_8UC1);
//only take the maxDist of each row
for(unsigned int y=0; y<dt.rows; ++y)
{
    float biggestDist = 0;
    cv::Point2i biggestDistLoc(0,0);
    for(unsigned int x=0; x<dt.cols; ++x)
    {
        cv::Point2i current(x,y);
        if(dt.at<float>(current) > biggestDist)
        {
            biggestDist = dt.at<float>(current) ;
            biggestDistLoc = current;

        }
    }
    lines.at<unsigned char>(biggestDistLoc) = 255;
}

//and the maxDist of each row
for(unsigned int x=0; x<dt.cols; ++x)
{
    float biggestDist = 0;
    cv::Point2i biggestDistLoc(0,0);
    for(unsigned int y=0; y<dt.rows; ++y)
    {
        cv::Point2i current(x,y);
        if(dt.at<float>(current) > biggestDist)
        {
            biggestDist = dt.at<float>(current) ;
            biggestDistLoc = current;

        }
    }
    lines.at<unsigned char>(biggestDistLoc) = 255;
}

cv::imshow("max", lines);


cv::waitKey(-1);

the idea is to compute the distance transform and find the ridges within the contour.

this is how the distance transformed image looks like: you can see that there is a local ridge maximum in the middle of the lines.

then I use a very simple method: just find the maximum distance in each row/column. That is very sloppy and should be changed for a real ridge detection or thinning method!!!!

edit: additional description:

The idea is to find all points that are in the "middle" of the contour. In mathematics/graphics, the medial axis in some kind of "middle" of an object and it's definition is to be all points that have the same minimum distance to at least two contour points at the same time.

A way to approximate the medial axis is to compute the distance transform. The distance transform is a matrix that holds for each pixel the distance to the next object point (for example a contour of an object)(see http://en.wikipedia.org/wiki/Distance_transform too). That's the first image. There you can see that the points in the middle of the lines are a little bit brighter than the points closer to the border, which means that the brightest points along the lines can be interpreted as the medial axis (approximation), since if you move away from it (orthogonal to the line direction) the distances become smaller, so the peak is the point where the distance to both borders in close to equality.

That way if you can find those "ridges" in the distance transform, you're done. Ridge detection is normally done by a Harris Operator (see http://en.wikipedia.org/wiki/Ridge_detection ).

In the fast and dirty version that I've posted, I try to detect the ridge by accepting only the maximum value in each line and in each row. That's ok for most horizontal and vertical ridges, but will fail for diagonal ones. So maybe you really want to exchange the for loops with a real ridge detection.

Answer 2

Interesting task :) Here is my solution:

Here is code:

#include <iostream>
#include <vector>
#include <stdio.h>
#include <stdarg.h>
#include <set>
#include "opencv2/opencv.hpp"
#include "fstream"
#include "iostream"
using namespace std;
using namespace cv;


int Thinning(unsigned char * ucBinedImg, unsigned char * ucThinnedImage, long lWidth, long lHeight, long lIterativeLimit)
{
    if(ucBinedImg == NULL)
        return -1;

    if(ucThinnedImage == NULL)
        return -2;

    if(lIterativeLimit == -1)
        lIterativeLimit = 60000;

    unsigned char x1, x2, x3, x4, x5, x6, x7, x8, xp;
    unsigned char g1, g2, g3, g4;
    unsigned char b1, b2, b3, b4;
    unsigned char np1, np2, npm;
    unsigned char *pUp, *pDown, *pImg;
    long    lDeletedPoints = 0;

    // set border 
    memcpy(ucThinnedImage, ucBinedImg, lWidth*lHeight);

    for(long it=0; it<lIterativeLimit; it++)
    {
        lDeletedPoints = 0;
        for(long i=1; i<lHeight-1; i++)
        {
            // init neighborhood
            pUp = ucBinedImg + (i-1)*lWidth;
            pImg = ucBinedImg + i*lWidth ;
            pDown = ucBinedImg + (i+1)*lWidth ;

            for( long j=1; j<lWidth-1; j++)
            {
                pUp++;
                pImg++;
                pDown++;

                if(!*pImg)
                    continue;

                x6 = *(pUp-1);
                x5 = *(pImg-1);
                x4 = *(pDown-1);

                x7 = *pUp;
                xp = *pImg;
                x3 = *pDown;

                x8 = *(pUp+1);
                x1 = *(pImg + 1);
                x2 = *(pDown + 1);

                b1 = !x1 && (x2 == 1 || x3 == 1);
                b2 = !x3 && (x4 == 1 || x5 == 1);
                b3 = !x5 && (x6 == 1 || x7 == 1);
                b4 = !x7 && (x8 == 1 || x1 == 1);

                g1 = (b1 + b2 + b3 + b4) == 1;

                np1 = x1|| x2;
                np1 += x3 || x4;
                np1 += x5 || x6;
                np1 += x7 || x8;
                np2 = x2|| x3;
                np2 += x4 || x5;
                np2 += x6 || x7;
                np2 += x8 || x1;

                npm = np1>np2?np2:np1;
                g2 = npm>=2 && npm<=3;

                g3 = (x1 && (x2 || x3 || !x8)) == 0;
                g4 = (x5 && (x6 || x7 || !x4)) == 0;

                // first part
                if(g1 && g2 && g3)
                {
                    // delete this point
                    ucThinnedImage[lWidth*i + j] = 0;
                    ++lDeletedPoints;
                }
            }

        }

        //syn
        memcpy(ucBinedImg, ucThinnedImage, lWidth*lHeight);

        for(long i=1; i<lHeight-1; i++)
        {
            // init neighborhood
            pUp = ucBinedImg + (i-1)*lWidth;
            pImg = ucBinedImg + i*lWidth ;
            pDown = ucBinedImg + (i+1)*lWidth ;

            for( long j=1; j<lWidth-1; j++)
            {
                pUp++;
                pImg++;
                pDown++;

                if(!*pImg)
                    continue;

                x6 = *(pUp-1);
                x5 = *(pImg-1);
                x4 = *(pDown-1);

                x7 = *pUp;
                xp = *pImg;
                x3 = *pDown;

                x8 = *(pUp+1);
                x1 = *(pImg + 1);
                x2 = *(pDown + 1);

                b1 = !x1 && (x2 == 1 || x3 == 1);
                b2 = !x3 && (x4 == 1 || x5 == 1);
                b3 = !x5 && (x6 == 1 || x7 == 1);
                b4 = !x7 && (x8 == 1 || x1 == 1);

                g1 = (b1 + b2 + b3 + b4) == 1;

                np1 = x1|| x2;
                np1 += x3 || x4;
                np1 += x5 || x6;
                np1 += x7 || x8;
                np2 = x2|| x3;
                np2 += x4 || x5;
                np2 += x6 || x7;
                np2 += x8 || x1;

                npm = np1>np2?np2:np1;
                g2 = npm>=2 && npm<=3;

                g3 = (x1 && (x2 || x3 || !x8)) == 0;
                g4 = (x5 && (x6 || x7 || !x4)) == 0;

                // second part
                if(g1 && g2 && g4)
                {
                    // delete this point
                    ucThinnedImage[lWidth*i + j] = 0;
                    ++lDeletedPoints;
                }

            }

        }
        //syn
        memcpy(ucBinedImg, ucThinnedImage, lWidth*lHeight);

        // if no points to be deleted
        if(lDeletedPoints == 0)
            break;

    }

    // clear edge bar
    for(long i=0; i<lHeight; i++)
    {
        for(long j=0; j<lWidth; j++)
        {
            if(i<16)
                ucThinnedImage[i*lWidth+j] = 0;
            else if(i>=lHeight-16)
                ucThinnedImage[i*lWidth+j] = 0;
            else if(j<16)
                ucThinnedImage[i*lWidth+j] = 0;
            else if(j>=lWidth-16)
                ucThinnedImage[i*lWidth+j] = 0;
        }
    }

    return 0;
}

void Thinning(Mat& src,Mat& dst,long IterativeLimit=-1)
{
    Mat bin_img=src&1;
    if(!dst.empty()){dst.release();}
    dst=Mat::zeros(src.size(),CV_8UC1);
    Thinning(bin_img.data,dst.data,bin_img.cols,bin_img.rows,IterativeLimit);
    dst*=255;
}

int main(int argc, char* argv[])
{
    namedWindow("source");
    namedWindow("result");

    Mat img=imread("raw_image.jpg",0);
    cv::threshold(img,img,128,255,cv::THRESH_BINARY);

    int erosion_size=5;
    Mat element = getStructuringElement( cv::MORPH_ELLIPSE,Size( 2*erosion_size + 1, 2*erosion_size+1 ),Point( erosion_size, erosion_size ) );

    cv::dilate(img,img,element);

    Mat thinned;
    Thinning(img,thinned);

    vector<Vec2f> lines;
    HoughLines(thinned, lines, 0.5, CV_PI/360, 50, 0, 0 );

    float hist_theta[2]={0,0};
    float hist_rho[2]={0,0};
    float n[2]={0,0};
    for( size_t i = 0; i < lines.size(); i++ )
    {
        float rho = lines[i][0], theta = lines[i][1];
        if(fabs(theta-CV_PI/2)<CV_PI/4)
        {
            hist_theta[0]+=theta;
            hist_rho[0]+=rho;
            n[0]+=1;
        }else
        {
            hist_theta[1]+=theta;
            hist_rho[1]+=rho;
            n[1]+=1;
        }
    }



    for( size_t i = 0; i < 2; i++ )
    {
        float rho = hist_rho[i]/n[i], theta = hist_theta[i]/n[i];
        Point pt1, pt2;
        double a = cos(theta), b = sin(theta);
        double x0 = a*rho, y0 = b*rho;
        pt1.x = cvRound(x0 + 1000*(-b));
        pt1.y = cvRound(y0 + 1000*(a));
        pt2.x = cvRound(x0 - 1000*(-b));
        pt2.y = cvRound(y0 - 1000*(a));
        line( thinned, pt1, pt2, Scalar(255,255,255), 1, CV_AA);
    }

    imshow("source",img);
    imshow("result",thinned);
    cv::waitKey(0);
}

Here is a hack in this source it uses two 1D histograms for postprocessing. In real life application it should use 2D histogram for close lines averaging.