Detect centre and angle of rectangles in an image using Opencv

Question

I have an image as below :

I need to find out the number of rectangles,centre of each rectangle and the measure the angle between the axis parallel to the longer edge of the rectangle passing through centre and measure the angle in anti­clockwise direction from the horizontal.I found out the number of rectangles in the image.I'm struck in finding out the centre and angle of reflection.Finding the centre through moments is not giving me the correct answer.

My code :

import cv2
import numpy as np 
import sys

img = cv2.imread(str(sys.argv[1]),0)
ret,thresh = cv2.threshold(img,127,255,0)
contours,hierarchy = cv2.findContours(thresh,1,2)



for contour in contours:
    area = cv2.contourArea(contour)
    if area>100000:
        contours.remove(contour)




cnt = contours[0]

epsilon = 0.02*cv2.arcLength(cnt,True)
approx = cv2.approxPolyDP(cnt,epsilon,True)

print 'No of rectangles',len(approx)


#finding the centre of the contour
M = cv2.moments(cnt)

cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])

print cx,cy

Answer 1

This is how you can do it with minAreaRect function of openCV. It's written in C++ but probably you can adapt that easily, since nearly only OpenCV functions were used.

    cv::Mat input = cv::imread("../inputData/rectangles.png");

    cv::Mat gray;
    cv::cvtColor(input,gray,CV_BGR2GRAY);

    // since your image has compression artifacts, we have to threshold the image
    int threshold = 200;
    cv::Mat mask = gray > threshold;

    cv::imshow("mask", mask);

    // extract contours
    std::vector<std::vector<cv::Point> > contours;
    cv::findContours(mask, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);

    for(int i=0; i<contours.size(); ++i)
    {
        // fit bounding rectangle around contour
        cv::RotatedRect rotatedRect = cv::minAreaRect(contours[i]);

        // read points and angle
        cv::Point2f rect_points[4]; 
        rotatedRect.points( rect_points );

        float  angle = rotatedRect.angle; // angle

        // read center of rotated rect
        cv::Point2f center = rotatedRect.center; // center

        // draw rotated rect
        for(unsigned int j=0; j<4; ++j)
            cv::line(input, rect_points[j], rect_points[(j+1)%4], cv::Scalar(0,255,0));

        // draw center and print text
        std::stringstream ss;   ss << angle; // convert float to string
        cv::circle(input, center, 5, cv::Scalar(0,255,0)); // draw center
        cv::putText(input, ss.str(), center + cv::Point2f(-25,25), cv::FONT_HERSHEY_COMPLEX_SMALL, 1, cv::Scalar(255,0,255)); // print angle
    }

resulting in this image:

as you can see, the angles are probably not what you want (because they randomly use the longer or the smaller line as reference). You can instead extract the longer sides of the rectangles and compute the angle manually.

If you choose the longer edge of the rotated rects and compute the angle from it it looks like this:

// choose the longer edge of the rotated rect to compute the angle
        cv::Point2f edge1 = cv::Vec2f(rect_points[1].x, rect_points[1].y) - cv::Vec2f(rect_points[0].x, rect_points[0].y);
        cv::Point2f edge2 = cv::Vec2f(rect_points[2].x, rect_points[2].y) - cv::Vec2f(rect_points[1].x, rect_points[1].y);

        cv::Point2f usedEdge = edge1;
        if(cv::norm(edge2) > cv::norm(edge1))
            usedEdge = edge2;

        cv::Point2f reference = cv::Vec2f(1,0); // horizontal edge


        angle = 180.0f/CV_PI * acos((reference.x*usedEdge.x + reference.y*usedEdge.y) / (cv::norm(reference) *cv::norm(usedEdge)));

giving this result, which should be what you are looking for!

EDIT: It looks like the op doesn't use the input image he posted, because reference rectangle centres would lie outside of the image.

Using this input (manually rescaled but probably still not optimal):

I get those results (blue dots are reference rectangle centers provided by the op):

Comparing the reference with the detections:

reference (x,y,angle)    detection (x,y,angle)
(320,240,0)              (320, 240, 180) // angle 180 is equal to angle 0 for lines
(75,175,90)              (73.5, 174.5, 90)
(279,401,170)            (279.002, 401.824, 169.992)
(507,379,61)             (507.842, 379.75, 61.1443)
(545,95,135)             (545.75, 94.25, 135)
(307,79,37)              (306.756, 77.8384, 37.1042)

I would love to see the REAL input image though, maybe the result will be even better.

Answer 2

Here is how you can do it:

1. Connected component labeling in order to detect each pattern (in your case the rectangles)
2. Separate the patterns in different images
3. (optional) if the pattern are not all rectangles, then use shape indexes to discriminate them
4. Compute the main axis using Principal Component Analysis (PCA), it will give you the angle you are looking for.