Deskewing MNIST dataset images using minAreaRect() of opencv

Question

I used opencv's minAreaRect to deskew the mnist digits.It worked well for most of the digits but,in some cases the minAreaRect was not detected correctly and it lead to further skewing of the digits.

Images with which this code worked:
Input image:
minAreaRect Image:
deskewed image:

But,for this the didn't work well:
Input image: minAreaRect Image: deskewed image:

I want to mention here that I did use: #coords = np.column_stack(np.where(thresh>0)) but,this didn't work at all. Please suggest a solution using minAreaRect(Preferred) function of opencv. And I've tested with several images and I do understand that the problem is with the formation of the min Area Rectangle,in the second example it is clear that the min Area rectangle is not visible(because it passess through the digit itself).

Here goes the code:

import numpy as np
import cv2

image=cv2.imread('MNIST/mnist_png/testing/9/73.png')#for 4##5032,6780 #8527,2436,1391
gray=cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)## for 9 problem with 4665,8998,73,7
gray=cv2.bitwise_not(gray)
Gblur=cv2.blur(gray,(5,5))
thresh=cv2.threshold(Gblur,0,255,cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]

#cv2.imshow("gray_thresh_blur",thresh)

#Finding Contours will be used to draw the min area rectangle

 _,contours,_=cv2.findContours(thresh.copy(),cv2.RETR_LIST,cv2.CHAIN_APPROX_NONE)
cnt1 = contours[0]
cnt=cv2.convexHull(contours[0])
angle = cv2.minAreaRect(cnt)[-1]
print("Actual angle is:"+str(angle))
rect = cv2.minAreaRect(cnt)

p=np.array(rect[1])
#print(p[0])
if p[0] < p[1]:
        print("Angle along the longer side:"+str(rect[-1] + 180))
        act_angle=rect[-1]+180
else:
        print("Angle along the longer side:"+str(rect[-1] + 90))
        act_angle=rect[-1]+90
#act_angle gives the angle with bounding box

if act_angle < 90:
        angle = (90 + angle)
        print("angleless than -45")

        # otherwise, just take the inverse of the angle to make
        # it positive
else:
        angle=act_angle-180
        print("grter than 90")

# rotate the image to deskew it
(h, w) = image.shape[:2]
center = (w // 2, h // 2)
M = cv2.getRotationMatrix2D(center, angle, 1.0)
rotated = cv2.warpAffine(image,M,(w,h),flags=cv2.INTER_CUBIC,borderMode=cv2.BORDER_REPLICATE)




box = cv2.boxPoints(rect)
print(box)
box = np.int0(box)
print(box)


p=cv2.drawContours(thresh,[box],0,(0,0,255),1)
print("contours"+str(p))
cv2.imwrite("post/MinAreaRect9.png",p)

cv2.imwrite("post/Input_9.png", image)
cv2.imwrite('post/Deskewed_9.png', rotated)

Answer 1

A few points to take note of:

• Most of OpenCV's functions work with white foreground and black background. So comment out this line:

gray=cv2.bitwise_not(gray)

• Make sure you're computing the EXTERNEL contours of the letters. This means that you need to ignore all the child contours. For this use cv2.RETR_EXTERNAL.

contours=cv2.findContours(thresh.copy(),cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)[1]

• Finally make sure you're assigning the correct angle to find rotation matrix.

With these changes: