Find rectangles without corners using opencv

Question

I have an image where I want to find contours but the "contours" in my image don't have corners. Are there some tricks I can use to help find the rectangles that are implied by the lines in this image? I thought about extending all the lines to form the corners but I worry about lines intersecting from other contours and how to determine which intersections I'm interested in. I'm very new to opencv and I don't know much about image processing. Thank you for any help you can give.

Answer 1

Fit lines in your binary image with the Hough transform and fit rectangles to the orthogonally intersecting lines.

Answer 2

I ended up implementing my own solution. It isn't very graceful but it gets the job done. I would be interested in hearing about improvements. HoughLines2 didn't always give me good results for finding line segments and I had to mess around with the threshold value a lot for different scenarios. Instead I opted for FindCountours where I took contours with two elements, I should be guaranteed 1 pixel wide lines. After finding the lines I iterated through them and traced them out to find the rectangles.

Where points is a *CvSeq of the line endpoints

while(points->total>0){
  if(p1.x==-1&&p1.y==-1){
     cvSeqPopFront(points,&p1);
     cvSeqPopFront(points,&p2);
  }

  if((pos=findClosestPoint(&p1,&p2, points,maxDist))>=0){  
     p3 = (CvPoint*)cvGetSeqElem( points,pos );
     pos2 = (pos%2==0)?pos+1:pos-1; //lines are in pairs of points
     p4 = (CvPoint*)cvGetSeqElem( points,pos2 );

     if(isVertical(&p1,&p2) && isHorizontal(p3,p4)){
        printf("found Corner %d %d\n",p2.x,p3->y);
     } else if(isHorizontal(&p1,&p2) && isVertical(p3,p4) ){
        printf("found Corner %d %d\n",p3->x,p2.y);
     }

     memcpy(&p1,p3,sizeof(CvPoint));
     memcpy(&p2,p4,sizeof(CvPoint));
     cvSeqRemove(points, (pos>pos2)?pos:pos2);
     cvSeqRemove(points, (pos>pos2)?pos2:pos);
  } else {
     p1.x=-1;
     p1.y=-1;
  }
}

int findClosestPoint (CvPoint *p1, CvPoint *p2, CvSeq *points, int maxDist) {  
   int ret = -1,i;
   float dist, minDist = maxDist;
   CvPoint* test;
   int (*dirTest)(CvPoint *,CvPoint *);

   if(isVertical(p1,p2)){ //vertical line
      if(p2->y > p1->y) {//going down
         dirTest = isBelow;
      } else { // going up
         dirTest = isAbove;
      }
   } else if (isHorizontal(p1,p2)){ //horizontal line
      if(p2->x > p1->x) {//going right
         dirTest = isRight;
      } else { //going left
         dirTest = isLeft;
      }
   }

   for( i = 0; i < points->total; i++ )
   {
      test = (CvPoint*)cvGetSeqElem( points, i );
      if(dirTest(p2,test)){ //only test points in the region we care about
         dist = sqrt(pow(test->x - p2->x,2)+pow(test->y - p2->y,2));
         if(dist<minDist){
            minDist = dist;
            ret = i;
         }
      }
   } 
   return ret;
}

int isVertical(CvPoint *p1, CvPoint *p2){
   return p1->x == p2->x;
}
int isHorizontal(CvPoint *p1, CvPoint *p2){
   return p1->y == p2->y;
}
int isRight(CvPoint *pt1, CvPoint *pt2){
   return pt2->x > pt1->x;
}
int isLeft(CvPoint *pt1, CvPoint *pt2){
   return pt2->x < pt1->x;
}
int isBelow(CvPoint *pt1, CvPoint *pt2){
   return pt2->y > pt1->y;
}
int isAbove(CvPoint *pt1, CvPoint *pt2){
   return pt2->y < pt1->y;
}

Answer 3

You could also try posing it as optimization problem. Rectangle is defined as 4D state vector (x,w,width,height) or 5D vector if you include rotation (x,y,width,height,rotation). For your current state you could do a gradient descent towards result of Hough lines to converge to the optimal state. Other option is using linear least squares: http://people.inf.ethz.ch/arbenz/MatlabKurs/node88.html