Text Extraction from image after detecting text region with contours

Question

I want to build an OCR for an image using machine learning in python. I have preprocessed image by converting it to grayscale , applied otsu thresholding . I then used the contours to find the text regions and draw rectangular boxes over it . But how do i extract the detected text after that . I dont want to use pytesseract . I want to use knn or SVM or CNN for prediction But The main problem I am facing is to how to get the detected text from image using contours .

Image=cv2.imread('DL.png')
I=Image.copy()
i=Image.copy()
G_Image=cv2.cvtColor(Image,cv2.COLOR_BGR2GRAY)

#Otsu Thresholding
blur = cv2.GaussianBlur(G_Image,(1,1),0)
ret,th = cv2.threshold(blur,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
image, contours, hierarchy = cv2.findContours(th,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
#img = cv2.drawContours(Image, contours, -1, (0,255,0), 3)

for contour in contours:
        # get rectangle bounding contour
        [x, y, w, h] = cv2.boundingRect(contour)

        if h>20:
            continue

        # draw rectangle around contour on original image
        cv2.rectangle(I, (x, y), (x + w, y + h), (255, 0, 255), 0)

Above is the code i wrote . This is the output image after contour rectangles are formed on detected text

Now how do I only use these detected regions and send them to my machine learning algorithm (KNN,SVM or CNN) for getting the text from image .

Answer 1

To crop the text areas you can use numpy slicing (as the image is effectively a numpy array):

letter = I[y:y+h, x:x+w]

In your loop that could create a new numpy array (cropped image) for every letter. Resize each of these to e.g. 28x28 and you have the right shape for the popular MNIST example.

For further ideas I can recommend the following git-repo, which creates a ML model for handwritten letters: EMNIST

It will be interesting how you handle incorrect/too coarse grained text detections like the "DE" or "RT" in DEPARTMENT. Andrew NG suggested in his Coursera Course for Machine Learning to use a ML model to detect the gaps between letters and split by these.