Reading colors of a rubik's cube using Opencv

Question

I have written a C++ program using OpenCV that can detect and highlight the edges of any object from a live video. But now I don't know how to extract the four corners of the cube from the many edges that are being detected in the video. So I am looking for some help here.

Here is the link of the paper that I am using as a guide for my this project. http://www.cs.ubc.ca/~andrejk/525project/525report.pdf You can find the program code for this paper in the link below. It's written in Python. (I am using C++ and I don't know Python) http://www.cs.ubc.ca/~andrejk/525project/cubefinder.py

According to the paper the next step would be, 'edge segmentation with adaptive threshold.' Which I don't really understand. And also I don't know how to extract the corners of the cube then.

The short summary of the method that I have used is as following. 1. Input from webcam 2. Apply Laplacian filter 3. Apply Hough Line Transform.

I get the following result.

Code

using namespace std;
using namespace cv;

Mat laplacianFilter(Mat image)
{
Mat hImage;

GaussianBlur(image,hImage,Size(3,3),0,0,BORDER_DEFAULT);
cvtColor(hImage,hImage,CV_RGB2GRAY);
Laplacian(hImage,hImage,CV_16SC1,3,1,0,BORDER_DEFAULT);
convertScaleAbs(hImage,hImage,1,0);

return hImage;
}

Mat hghTransform(Mat image, Mat &image2)
{
Mat lImage;

Canny(image,image,50,200,3);
cvtColor(image,lImage,CV_GRAY2BGR);

    vector<Vec4i> lines;
    HoughLinesP(image, lines, 1, CV_PI/180, 50, 50, 10 );
    for( size_t i = 0; i < lines.size(); i++ )
    {
        Vec4i l = lines[i];
        line( image2, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0,255,0), 3, CV_AA);
    }

return lImage;
}


int main()
{
int c;

VideoCapture cap(0);

Mat image;
Mat image2;

namedWindow("hghtransform");
namedWindow("laplacianfilter");
namedWindow("cannyOutput");

while(1)
{
    cap>>image;
    cap>>image2;

    //Output
    imshow("laplacianfilter",laplacianFilter(image));
    imshow("cannyOutput",hghTransform(laplacianFilter(image),image2));
    imshow("hghtransform",image2);

    c=waitKey(33);

    if(c==27)
        return 0;

}

return 0;
}

Answer 1

Adaptive threshold will give you a clear line of edges which enables you to get 9 squares of a rubik side properly.

You can see a decent comparison of global and adaptive threshold here:
here: https://sites.google.com/site/qingzongtseng/adaptivethreshold

original image:

global threshold:

adaptive threshold:

For the corner, I am not sure whether it's stated in the paper, but I would do something like:
==> finding area like 1, 2, 3, 4 for upper-left, upper-right, lower-left, and lower-right corner respectively
==> with a template matching algorithm.

hope it helps.

note: you might want to have a background with less noise there. =)