Remove glare from photo opencv

Question

So, im using opencv to capture a document, scan it and crop it. When there is no lighting in the room, it works perfectly. When there is some light in the room, and there is a glare on the table and the document is near it, it also grabs the glare as part of the rectangle.

How can one remove the glare from the photo?

Here is the code im using to get the image I want:

 Mat &image = *(Mat *) matAddrRgba;
    Rect bounding_rect;

    Mat thr(image.rows, image.cols, CV_8UC1);
    cvtColor(image, thr, CV_BGR2GRAY); //Convert to gray
    threshold(thr, thr, 150, 255, THRESH_BINARY + THRESH_OTSU); //Threshold the gray

    vector<vector<Point> > contours; // Vector for storing contour
    vector<Vec4i> hierarchy;
    findContours(thr, contours, hierarchy, CV_RETR_CCOMP,
                 CV_CHAIN_APPROX_SIMPLE); // Find the contours in the image
    sort(contours.begin(), contours.end(),
         compareContourAreas);            //Store the index of largest contour
    bounding_rect = boundingRect(contours[0]);

    rectangle(image, bounding_rect, Scalar(250, 250, 250), 5);

Here is a photo of the glare im talking about:

The things I have found are to use inRange, find the apropriate scalar for color and us inpaint to remove light. Here is a code snippet of that, but it always crashes saying it needs 8bit image with chanels.

Mat &image = *(Mat *) matAddrRgba;

    Mat hsv, newImage, inpaintMask;
    cv::Mat lower_red_hue_range;
    inpaintMask = Mat::zeros(image.size(), CV_8U);
    cvtColor(image, hsv, COLOR_BGR2HSV);
    cv::inRange(hsv, cv::Scalar(0, 0, 215, 0), cv::Scalar(180, 255, 255, 0),
                lower_red_hue_range);
    image = lower_red_hue_range;

    inpaint(image, lower_red_hue_range, newImage, 3, INPAINT_TELEA);

Answer 1

I have dealt with this problem before, and change in lighting is always a problem in Computer Vision for detection and description of images. I actually trained a classifier, for HSV color spaces instead of RGB/BGR, which was mapping the image with changing incident light to the one which doesn't have the sudden brightness/dark patches (this would be the label). This worked for me quite well, however, the images were always of the same background (I don't know if you also have this).

Of course, machine learning can solve the problem but it might be an overkill. While I was doing the above mentioned, I came across CLAHE which worked pretty well with for local contrast enhancement. I suggest you to try this before detecting contours. Additionally, you might want to work on a different color space, such as HSV/Lab/Luv instead of RGB/BGR for this purpose. You can apply CLAHE separately to each channel and then merge them.

Let me know if you need some other information. I implemented this with your image in python, it works pretty nicely, but I would leave the coding to you. I might update the results I got after a couple of days (hoping that you get them first ;) ). Hope it helps.

Answer 2

opencv-python helpers

 import cv2
 import numpy as np
 import time


clahefilter = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(16,16))


img = cv2.imread('spects_glare.jpg')

while True:
t1 = time.time() 
img = img.copy()

## crop if required 
#FACE
x,y,h,w = 550,250,400,300
# img = img[y:y+h, x:x+w]

#NORMAL
# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
grayimg = gray


GLARE_MIN = np.array([0, 0, 50],np.uint8)
GLARE_MAX = np.array([0, 0, 225],np.uint8)

hsv_img = cv2.cvtColor(img,cv2.COLOR_BGR2HSV)

#HSV
frame_threshed = cv2.inRange(hsv_img, GLARE_MIN, GLARE_MAX)


#INPAINT
mask1 = cv2.threshold(grayimg , 220, 255, cv2.THRESH_BINARY)[1]
result1 = cv2.inpaint(img, mask1, 0.1, cv2.INPAINT_TELEA) 



#CLAHE
claheCorrecttedFrame = clahefilter.apply(grayimg)

#COLOR 
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
lab_planes = cv2.split(lab)
clahe = cv2.createCLAHE(clipLimit=2.0,tileGridSize=(8,8))
lab_planes[0] = clahe.apply(lab_planes[0])
lab = cv2.merge(lab_planes)
clahe_bgr = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR)


#INPAINT + HSV
result = cv2.inpaint(img, frame_threshed, 0.1, cv2.INPAINT_TELEA) 


#INPAINT + CLAHE
grayimg1 = cv2.cvtColor(clahe_bgr, cv2.COLOR_BGR2GRAY)
mask2 = cv2.threshold(grayimg1 , 220, 255, cv2.THRESH_BINARY)[1]
result2 = cv2.inpaint(img, mask2, 0.1, cv2.INPAINT_TELEA) 



#HSV+ INPAINT + CLAHE
lab1 = cv2.cvtColor(result, cv2.COLOR_BGR2LAB)
lab_planes1 = cv2.split(lab1)
clahe1 = cv2.createCLAHE(clipLimit=2.0,tileGridSize=(8,8))
lab_planes1[0] = clahe1.apply(lab_planes1[0])
lab1 = cv2.merge(lab_planes1)
clahe_bgr1 = cv2.cvtColor(lab1, cv2.COLOR_LAB2BGR)




# fps = 1./(time.time()-t1)
# cv2.putText(clahe_bgr1    , "FPS: {:.2f}".format(fps), (10, 180), cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255))    

# display it
cv2.imshow("IMAGE", img)
cv2.imshow("GRAY", gray)
cv2.imshow("HSV", frame_threshed)
cv2.imshow("CLAHE", clahe_bgr)
cv2.imshow("LAB", lab)
cv2.imshow("HSV + INPAINT", result)
cv2.imshow("INPAINT", result1)
cv2.imshow("CLAHE + INPAINT", result2)  
cv2.imshow("HSV + INPAINT + CLAHE   ", clahe_bgr1)


# Break with esc key
if cv2.waitKey(1) & 0xFF == ord('q'): 
    break


cv2.destroyAllWindows()