Image Registration of Scanned Text Forms

Question

We print 500 bubble surveys, get them back, and scan them in a giant batch giving us 500 PNG images.

Each image has a slight variations in alignment, but identical size and resolution. We need to register the images so they're all perfectly aligned. (With the next step being semi-automated scoring of the bubbles).

If these were 3D-MRI images, I could accomplish this with a single command line utility; But I'm not seeing any such tool for aligning scanned text documents.

I've played around with opencv as described in Image Alignment (Feature Based) using OpenCV, and it produces dynamite results when it works, but it often fails spectacularly. That approach is looking for documents hidden within natural scenes, a much harder problem than our case where the images are just rotated and translated in 2D, not 3.

I've also explored imreg_dft, which runs consistently but does a very poor job -- presumably the dft approach is better on photographs than text documents.

Does a solution for Image Registration of Scanned Forms already exist? If not, what's the correct approach? Opencv, imreg_dft, or something else?

Similar prior question: How to find blank field on scanned document image

Answer 1

What you can try is using the red outline of the answer boxes to create a mask where you can select the outline. I create a sample below. You can also remove the blue letters by creating a mask for the letters, inverting it, then apply it as a mask. I didn't do that, because he image of the publisher is low-res, and it caused issues. I expect your scans to perform better.

When you have the contours of the boxes you can transform/compare them individually (as the boxes have different sizes). Or you can use the biggest contour to create a transform for the entire document.

You can then use minAreaRect to find the cornerpoints of the contours. Threshold the contourArea to exclude noise / non answer area's.

import cv2
import numpy as np
# load image
img = cv2.imread('Untitled.png')
# convert to hsv colorspace
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# define range of image bachground in HSV
lower_val = np.array([0,0,0])
upper_val = np.array([179,255,237])

# Threshold the HSV image 
mask = cv2.inRange(hsv, lower_val, upper_val)

# find external contours in the mask
contours, hier = cv2.findContours(mask, cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
# draw contours
for cnt in contours:
    cv2.drawContours(img,[cnt],0,(0,255,0),3)

# display image
cv2.imshow('Result', img)
cv2.waitKey(0)
cv2.destroyAllWindows()