Detect small rectangles in image AForge

Question

I'm trying to detect rectangles on this image:

with this code:

static void Main(string[] args)
{
    // Open your image
    string path = "test.png";
    Bitmap image = (Bitmap)Bitmap.FromFile(path);

    // locating objects
    BlobCounter blobCounter = new BlobCounter();

    blobCounter.FilterBlobs = true;
    blobCounter.MinHeight = 5;
    blobCounter.MinWidth = 5;

    blobCounter.ProcessImage(image);
    Blob[] blobs = blobCounter.GetObjectsInformation();

    // check for rectangles
    SimpleShapeChecker shapeChecker = new SimpleShapeChecker();

    foreach (var blob in blobs)
    {
        List<IntPoint> edgePoints = blobCounter.GetBlobsEdgePoints(blob);
        List<IntPoint> cornerPoints;

        // use the shape checker to extract the corner points
        if (shapeChecker.IsQuadrilateral(edgePoints, out cornerPoints))
        {
            // only do things if the corners form a rectangle
            if (shapeChecker.CheckPolygonSubType(cornerPoints) == PolygonSubType.Rectangle)
            {
                // here i use the graphics class to draw an overlay, but you
                // could also just use the cornerPoints list to calculate your
                // x, y, width, height values.
                List<Point> Points = new List<Point>();
                foreach (var point in cornerPoints)
                {
                    Points.Add(new Point(point.X, point.Y));
                }

                Graphics g = Graphics.FromImage(image);
                g.DrawPolygon(new Pen(Color.Red, 5.0f), Points.ToArray());

                image.Save("result.png");
            }
        }
    }
}

but it dont recognize the rectangles (walls). It just recognize the big square, and when I reduce the minHeight and minWidth, it recognize trapezoids on the writing..

Answer 1

I propose a different algorithm approach, after working almost a year with image processing algorithms what I can tell is that to create an efficient algorithm, you have to "reflect" how you, as a human would do that, here is the proposed approach:

1. We don't really care about the textures, we care about the edges (rectangles are edges), therefore we will apply an Edge-detection>Difference (http://www.aforgenet.com/framework/docs/html/d0eb5827-33e6-c8bb-8a62-d6dd3634b0c9.htm), this gives us:

2. We want to exaggerate the walls, as humans we know that we are looking for the walls, but the computer does not know this, therefore, apply two rounds of Morphology>Dilatation (http://www.aforgenet.com/framework/docs/html/88f713d4-a469-30d2-dc57-5ceb33210723.htm), this gives us:

3. We care only about the what is wall and what is not, apply a Binarization>Threshold (http://www.aforgenet.com/framework/docs/html/503a43b9-d98b-a19f-b74e-44767916ad65.htm), we get:

4. (Optional) We can apply a blob extraction to erase the labels ("QUARTO, BANHEIRO", etc)

5. We apply a Color>Invert, this is just done because the next step detects the white color not black.

6. Apply a Blob>Processing>Connected Components Labeling (http://www.aforgenet.com/framework/docs/html/240525ea-c114-8b0a-f294-508aae3e95eb.htm), this will give us all the rectangles, like this:

Note that for each colored box you have its coordinates, center, width and height. So you can extract a snip from the real image with that coordinates.

PS: Using the program AForge Image Processing Lab is highly recommended to test your algos.