Image Processing Edge Detection in Java

Question

This is my situation. It involves aligning a scanned image which will account for incorrect scanning. I must align the scanned image with my Java program.

These are more details:

• There is a table-like form printed on a sheet of paper, which will be scanned into an image file.
• I will open the picture with Java, and I will have an OVERLAY of text boxes.
• The text boxes are supposed to align correctly with the scanned image.
• In order to align correctly, my Java program must analyze the scanned image and detect the coordinates of the edges of the table on the scanned image, and thus position the image and the textboxes so that the textboxes and the image both align properly (in case of incorrect scanning)

You see, the guy scanning the image might not necessarily place the image in a perfectly correct position, so I need my program to automatically align the scanned image as it loads it. This program will be reusable on many of such scanned images, so I need the program to be flexible in this way.

My question is one of the following:

1. How can I use Java to detect the y coordinate of the upper edge of the table and the x-coordinate of the leftmost edge of the table. The table is a a regular table with many cells, with black thin border, printed on a white sheet of paper (horizontal printout)

2. If an easier method exists to automatically align the scanned image in such a way that all scanned images will have the graphical table align to the same x, y coordinates, then share this method :).

3. If you don't know the answer to the above to questions, do tell me where I should start. I don't know much about graphics java programming and I have about 1 month to finish this program. Just assume that I have a tight schedule and I have to make the graphics part as simple as possible for me.

Cheers and thank you.

Answer 1

Try to start from a simple scenario and then improve the approach.

1. Detect corners.
2. Find the corners in the boundaries of the form.
3. Using the form corners coordinates, calculate the rotation angle.
4. Rotate/scale the image.
5. Map the position of each field in the form relative to form origin coordinates.
6. Match the textboxes.

The program presented at the end of this post does the steps 1 to 3. It was implemented using Marvin Framework. The image below shows the output image with the detected corners.

The program also outputs: Rotation angle:1.6365770416167182

Source code:

import java.awt.Color;
import java.awt.Point;
import marvin.image.MarvinImage;
import marvin.io.MarvinImageIO;
import marvin.plugin.MarvinImagePlugin;
import marvin.util.MarvinAttributes;
import marvin.util.MarvinPluginLoader;

public class FormCorners {

public FormCorners(){
    // Load plug-in
    MarvinImagePlugin moravec = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.corner.moravec");
    MarvinAttributes attr = new MarvinAttributes();

    // Load image
    MarvinImage image = MarvinImageIO.loadImage("./res/printedForm.jpg");

    // Process and save output image
    moravec.setAttribute("threshold", 2000);
    moravec.process(image, null, attr);
    Point[] boundaries = boundaries(attr);
    image = showCorners(image, boundaries, 12);
    MarvinImageIO.saveImage(image, "./res/printedForm_output.jpg");

    // Print rotation angle
    double angle =  (Math.atan2((boundaries[1].y*-1)-(boundaries[0].y*-1),boundaries[1].x-boundaries[0].x) * 180 / Math.PI);
    angle =  angle >= 0 ? angle : angle + 360;
    System.out.println("Rotation angle:"+angle);
}

private Point[] boundaries(MarvinAttributes attr){
    Point upLeft = new Point(-1,-1);
    Point upRight = new Point(-1,-1);
    Point bottomLeft = new Point(-1,-1);
    Point bottomRight = new Point(-1,-1);
    double ulDistance=9999,blDistance=9999,urDistance=9999,brDistance=9999;
    double tempDistance=-1;
    int[][] cornernessMap = (int[][]) attr.get("cornernessMap");

    for(int x=0; x<cornernessMap.length; x++){
        for(int y=0; y<cornernessMap[0].length; y++){
            if(cornernessMap[x][y] > 0){
                if((tempDistance = Point.distance(x, y, 0, 0)) < ulDistance){
                    upLeft.x = x; upLeft.y = y;
                    ulDistance = tempDistance;
                } 
                if((tempDistance = Point.distance(x, y, cornernessMap.length, 0)) < urDistance){
                    upRight.x = x; upRight.y = y;
                    urDistance = tempDistance;
                }
                if((tempDistance = Point.distance(x, y, 0, cornernessMap[0].length)) < blDistance){
                    bottomLeft.x = x; bottomLeft.y = y;
                    blDistance = tempDistance;
                }
                if((tempDistance = Point.distance(x, y, cornernessMap.length, cornernessMap[0].length)) < brDistance){
                    bottomRight.x = x; bottomRight.y = y;
                    brDistance = tempDistance;
                }
            }
        }
    }
    return new Point[]{upLeft, upRight, bottomRight, bottomLeft};
}

private MarvinImage showCorners(MarvinImage image, Point[] points, int rectSize){
    MarvinImage ret = image.clone();
    for(Point p:points){
        ret.fillRect(p.x-(rectSize/2), p.y-(rectSize/2), rectSize, rectSize, Color.red);
    }
    return ret;
}

public static void main(String[] args) {
    new FormCorners();
}
}