Detect chinese chess pieces in chessboard using OpenCV

Question

How can chinese chess pieces be detected in this image using OpenCV?

I've tried using HoughCircles, but no circle was found.

Mat src = imread( "x.jpg", CV_LOAD_IMAGE_GRAYSCALE);
GaussianBlur( src, src, Size(9, 9), 2, 2 );

vector<Vec3f> circles;
HoughCircles( src, circles, CV_HOUGH_GRADIENT, 1, src.rows/16);
cout << circles.size() << endl;
// The output is: 0

Blob detector was also tested, but the result was incorrect.

Mat im = imread( "x.jpg", IMREAD_GRAYSCALE );

vector<KeyPoint> kps;
SimpleBlobDetector().detect(im, kps);

Mat im_kps;
drawKeypoints( im, kps, im_kps, Scalar(0,0,255), DrawMatchesFlags::DRAW_RICH_KEYPOINTS );
imshow("keypoints", im_kps );
waitKey(0);

Answer 1

One problem that may arise when isolating the red colors from the image m is the mixing of the chess pieces with the also-red background. When running inRange(), this happens:

It would be so hard to tell where the checkers are at first glance! But we can use a trick clled dilation and erosion to remove the minor parts (the grid) while retaining the important parts (the circular checker).

Here is the code with dilation and erosion to solve the checker problem:

#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>

using namespace cv;
using namespace std;

Mat getElement(int erosion_size)
{
  return getStructuringElement(cv::MORPH_ELLIPSE,
         cv::Size(2 * erosion_size + 1, 2 * erosion_size + 1),
         cv::Point(erosion_size, erosion_size) );
}
int main( )
{
    vector<vector<Point> > contours;
    //vector<Vec4i> hierarchy;
    //int largest_area,largest_contour,largest_contour_index;
    //RotatedRect bounding_ellipse;

     Mat image,dst,filtered;
     Mat a1,a2;

     //Color ranging red (from both left & right spectrums)
     image = imread("circles.jpg");
     cvtColor(image,filtered,CV_BGR2HSV);

     Scalar low = Scalar(0, 100, 50);
     Scalar up = Scalar(10, 255, 255);

     Scalar low2 = Scalar(160, 100, 50);
     Scalar up2 = Scalar(179, 255, 255);

     inRange(filtered, low2, up2, a1);
     inRange(filtered, low2, up2, a2);
     bitwise_or(a1,a2,filtered);

     imshow("troll", filtered);


     // Fill in small holes from Chinese lettering
     dilate(filtered,filtered,getElement(11));

     imshow("better", filtered);
     // Erode to remove minor (like square lines) objects
     erode(filtered,filtered,getElement(25));
     imshow("best",filtered);

     findContours(filtered, contours, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);

     for( int i = 0; i< contours.size(); i++ )
     {
       //Only draw big contours, not minor details
        double area =contourArea( contours[i],false);
        if(area > 1500)
        {
          drawContours(image, contours, i, Scalar(0,255,0), 5);
        }
    }
     imshow( "Result window", image );

     waitKey(0);
     return 0;
}

Explanation:

1. Dilation

We start with dilation, which allows the brighter parts of the image to "expand" over the "darker" parts. So, in here, we will use this to remove the Chinese lettering (so when, in dilation, we will not have big holes in the middle of the circle):

2. Erosion

As you can see, now the circles are filled, and we can proceed on with erosion. We need to dilate more than the amount we eroded, because we need to remove the grid bars from the image. Applying erosion, we get only the checkers on the board (and some noise we will process later):

3. Contour processing

Now, we can process the checkers, but we need to filter out the noise around the image. To do this, we will run findContours() from our erosion results, but we will also check the area, which is contourArea(), of the contours to make sure it is our checkers. If the area is less than 1500, we know it is noise and we can trash it. Or else, we can draw it to the screen.