OpenCV find coloured in circle and position value Python

Question

What I'm trying to do is process the attendance sheet below to tell me who's there and who's not

I'm currently using matchTemplate using a singular black dot which finds all the filled in dots (image is first converted to grey scale). Image below

Then I get manipulate the matchPattern array and get approximately the center of each in the y direction and I can see where there are gaps corresponding to missing students.

The issue I am having is that sure this sorta works for perfect input but my goal is to be able to take a picture of the physical piece of paper and process this? Note: the attendance sheet is made by me so it can be changed/modified however necessary.

I have attached an example image to match seen below. Using my current method is just a disaster (seen below). Now I'm not sure where to go from here I tried modifying the threshold but past .65 it fails to find any images.

import cv2
import numpy as np
from matplotlib import pyplot as plt

values = []
img_rgb = cv2.imread('/home/user/Downloads/input.png')
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('/home/user/Downloads/input_2.png',0)
w, h = template.shape[::-1]

res = cv2.matchTemplate(img_gray,template,cv2.TM_CCOEFF_NORMED)
threshold = 0.6
loc = np.where( res >= threshold)
for pt in zip(*loc[::-1]):
  cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (0,0,255), 2)
  values.append(pt[1]+h/2)

cv2.imwrite('output.png',img_rgb)
values.sort()
pivot = values[0]
count = 1
total = values[0]
cleaned = []

for x in range(1,len(values)):
  if(values[x] < pivot+20):
    pivot = values[x]
    count = count + 1
    total = total + values[x]
  else:
    print values[x]
    cleaned.append(int(total/count))
    pivot = values[x]
    count = 1
    total = values[x]

  if x == len(values)-1:
    cleaned.append(int(total/count))
print values
print cleaned

Here is another test image:

Answer 1

Normally, when analyzing paper forms, special marks on margins and form corners are used to identify scale and orientation of meaningful parts. For example, you can print several little black squares on borders of the form, find them with the same cv2.matchTemplate and thus define region of interest.

Most probably your forms won't be perfectly captured (e.g. they may be scaled, rotated or seen in perspective), so you also need to normalize input. You may use perspective or affine transformations for this.

You may also want to enhance image by using histogram equalization, denoising and other techniques.

At this point you should have completely normalized image which is much closer to a "perfect input". You can try out your algorithm on this input, but there's also simpler way (AFAIK, something like this is used in real applications for automatic form analysis).

Your form has fixed layout and you already know its corners. So why not to calculate position of each interesting piece of the form? E.g. at the picture below I placed 4 black landmarks at the corners of the form. In a normalized image position of the dashed area will always be the same relative to these landmarks.

Finding out whether student was on the lecture or not is as simple as splitting dashed area into fixed square regions (one per student), summing pixel values in this area and comparing this value to a predefined threshold. Regions that have lower values tend to be more black than white (student attended the lecture), while regions with high values are most likely white (student was absent).

So, to sum up:

1. Use landmarks to define corners of the paper.
2. Normalize image with respect to these landmarks.
3. Enhance image if needed.
4. Calculate positions of regions of interest.
5. Determine if region is more likely black of white.

Answer 2

Here's an example of how to identify the dots with a fairly simplistic approach:

from skimage import io, color, filter, morphology, feature, img_as_float
from skimage.morphology import disk

image = io.imread('dots.jpg')

# Remove blue channel
bw = image.copy()
bw[..., 2] = 0

bw = 1 - img_as_float(color.rgb2gray(image))

big_mask = 150
small_mask = 10
min_dist = 50

bw = filter.rank.threshold_percentile(bw, disk(big_mask), p0=0.95)
bw = morphology.erosion(bw, disk(small_mask))

peaks = feature.corner_peaks(bw, min_distance=min_dist, indices=True)

import matplotlib.pyplot as plt
f, (ax0, ax1) = plt.subplots(1, 2)

ax0.imshow(image, cmap=plt.cm.gray)
ax1.imshow(bw, cmap=plt.cm.gray)
ax0.scatter(peaks[:, 1], peaks[:, 0], c='red', s=30)

plt.show()