How to prevent INPAINTING blocks and improve coloring

Question

I wanted to Remove all the texts USING INPAINTING from this IMAGE. I had been trying various methods, and eventually found that I can get the results through OCR and then using thresholding MASK THE IMAGE.

processedImage = preprocess(partOFimg) 
mask = np.ones(img.shape[:2], dtype="uint8") * 255
for c in cnts:
        cv2.drawContours(mask, [c], -1, 0, -1)
img = cv2.inpaint(img,mask,7,cv2.INPAINT_TELEA)

Preprocess operations:

  ret,thresh1 = cv2.threshold(gray, 0, 255,cv2.THRESH_OTSU|cv2.THRESH_BINARY_INV)
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15, 3))
dilation = cv2.dilate(thresh1, rect_kernel, iterations = 1)
edged = cv2.Canny(dilation, 50, 100)
cnts = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
mask =

np.ones(img.shape[:2], dtype="uint8") * 255

When I run the above code, I here am the OUTPUT Image OUTPUT. As we can see, it is making some BLOCKS OF DIFFERENT COLOR over the IMAGE, I want to prevent that, How do I achieve this? I see that mask images are not formed well many times, and in cases when the text is white the PREPROCESSING doesn't occur properly. How do I prevent these BLOCKS of other colours to FORM on the IMAGE?

Grayed Sub Image GRAYED

Threshold Sub IMG part: Thresholded Image

Masked Image Masked

EDIT 1:

I've managed to get this new better result by noticing that my threshold is the best mask I can get. After doing this I performed the masking process 3 different times with variable masks and inversions. I did the inpainting algorithm 3 times, it basically the other times inverse the mask, because in some cases required mask is the inversed mask. Still I think it needs improvement, If I chose a different image the results are not so good.

Answer 1

Python/OpenCV inpaint methods, generally, are not appropriate to your type of image. They work best on thin (scratch-like) regions, not large blocks. You really need an exemplar type method such as https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/criminisi_tip2004.pdf. But OpenCV does not have that.

However, the OpenCV methods do work here, I suspect, because you are filling with constant colors (green) and not texture. So you are best to try to get the mask of just the letters (characters), not rectangular blocks for the words. So, to show you what I mean, here is my Python/OpenCV approach.

Input:

• Read the input
• Threshold on the green sign
• Apply morphology to close it up and keep as mask1
• Apply the mask to the image to blacken out the outside of the sign
• Threshold on the white in this new image and keep as mask2
• Apply morphology dilate to enlarge it slightly and save as mask3
• Do the inpaint
• Save the results

import cv2
import numpy as np

# read input
img = cv2.imread('airport_sign.jpg')

# threshold on green sign
lower = (30,80,0)
upper = (70,120,20)
thresh = cv2.inRange(img, lower, upper)

# apply morphology close
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (135,135))
mask1 = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# apply mask to img
img2 = img.copy()
img2[mask1==0] = (0,0,0)

# threshold on white
#gray = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
#mask2 = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY+cv2.THRESH_OTSU)[1]
lower = (120,120,120)
upper = (255,255,255)
mask2 = cv2.inRange(img2, lower, upper)

# apply morphology dilate
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
mask3 = cv2.morphologyEx(mask2, cv2.MORPH_DILATE, kernel)


# do inpainting
result1 = cv2.inpaint(img,mask3,11,cv2.INPAINT_TELEA)
result2 = cv2.inpaint(img,mask3,11,cv2.INPAINT_NS)

# save results
cv2.imwrite('airport_sign_mask.png', mask3)
cv2.imwrite('airport_sign_inpainted1.png', result1)
cv2.imwrite('airport_sign_inpainted2.png', result1)

# show results
cv2.imshow('thresh',thresh)
cv2.imshow('mask1',mask1)
cv2.imshow('img2',img2)
cv2.imshow('mask2',mask2)
cv2.imshow('mask3',mask3)
cv2.imshow('result1',result1)
cv2.imshow('result2',result2)
cv2.waitKey(0)
cv2.destroyAllWindows()

Mask 3:

Inpaint 1 (Telea):

Inpaint 2 (NS):