OpenCV MSER detect text areas - Python

Question

I have an invoice image, and I want to detect the text on it. So I plan to use 2 steps: first is to identify the text areas, and then using OCR to recognize the text.

I am using OpenCV 3.0 in python for that. I am able to identify the text(including some non text areas) but I further want to identify text boxes from the image(also excluding the non-text areas).

My input image is: and the output is: and I am using the below code for this:

img = cv2.imread('/home/mis/Text_Recognition/bill.jpg') mser = cv2.MSER_create() gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #Converting to GrayScale gray_img = img.copy()  regions = mser.detectRegions(gray, None) hulls = [cv2.convexHull(p.reshape(-1, 1, 2)) for p in regions] cv2.polylines(gray_img, hulls, 1, (0, 0, 255), 2) cv2.imwrite('/home/mis/Text_Recognition/amit.jpg', gray_img) #Saving 

Now, I want to identify the text boxes, and remove/unidentify any non-text areas on the invoice. I am new to OpenCV and am a beginner in Python. I am able to find some examples in MATAB example and C++ example, but If I convert them to python, it will take a lot of time for me.

Is there any example with python using OpenCV, or can anyone help me with this?

Answer 1

Below is the code

# Import packages  import cv2 import numpy as np  #Create MSER object mser = cv2.MSER_create()  #Your image path i-e receipt path img = cv2.imread('/home/rafiullah/PycharmProjects/python-ocr-master/receipts/73.jpg')  #Convert to gray scale gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  vis = img.copy()  #detect regions in gray scale image regions, _ = mser.detectRegions(gray)  hulls = [cv2.convexHull(p.reshape(-1, 1, 2)) for p in regions]  cv2.polylines(vis, hulls, 1, (0, 255, 0))  cv2.imshow('img', vis)  cv2.waitKey(0)  mask = np.zeros((img.shape[0], img.shape[1], 1), dtype=np.uint8)  for contour in hulls:      cv2.drawContours(mask, [contour], -1, (255, 255, 255), -1)  #this is used to find only text regions, remaining are ignored text_only = cv2.bitwise_and(img, img, mask=mask)  cv2.imshow("text only", text_only)  cv2.waitKey(0) 

Answer 2

This is an old post, yet I'd like to contribute that if you are trying to extract all the texts out of an image, here is the code to get that text in an array.

import cv2 import numpy as np import re import pytesseract from pytesseract import image_to_string pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe" from PIL import Image  image_obj = Image.open("screenshot.png")  rgb = cv2.imread('screenshot.png') small = cv2.cvtColor(rgb, cv2.COLOR_BGR2GRAY)  #threshold the image _, bw = cv2.threshold(small, 0.0, 255.0, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)  # get horizontal mask of large size since text are horizontal components kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (20, 1)) connected = cv2.morphologyEx(bw, cv2.MORPH_CLOSE, kernel)  # find all the contours contours, hierarchy,=cv2.findContours(connected.copy(),cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) #Segment the text lines counter=0 array_of_texts=[] for idx in range(len(contours)):     x, y, w, h = cv2.boundingRect(contours[idx])     cropped_image = image_obj.crop((x-10, y, x+w+10, y+h ))     str_store = re.sub(r'([^\s\w]|_)+', '', image_to_string(cropped_image))     array_of_texts.append(str_store)     counter+=1  print(array_of_texts)