remove background of any image using opencv

Question

I have been searching for a technique to remove the background of a any given image. The idea is to detect a face and remove the background of the detected face. I have finished the face part. Now removing the background part still exists.

I used this code.

import cv2
import numpy as np

#== Parameters           
BLUR = 21
CANNY_THRESH_1 = 10
CANNY_THRESH_2 = 200
MASK_DILATE_ITER = 10
MASK_ERODE_ITER = 10
MASK_COLOR = (0.0,0.0,1.0) # In BGR format


#-- Read image
img = cv2.imread('SYxmp.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

#-- Edge detection 
edges = cv2.Canny(gray, CANNY_THRESH_1, CANNY_THRESH_2)
edges = cv2.dilate(edges, None)
edges = cv2.erode(edges, None)

#-- Find contours in edges, sort by area 
contour_info = []
contours, _ = cv2.findContours(edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)
for c in contours:
    contour_info.append((
        c,
        cv2.isContourConvex(c),
        cv2.contourArea(c),
    ))
contour_info = sorted(contour_info, key=lambda c: c[2], reverse=True)
max_contour = contour_info[0]

#-- Create empty mask, draw filled polygon on it corresponding to largest contour ----
# Mask is black, polygon is white
mask = np.zeros(edges.shape)
cv2.fillConvexPoly(mask, max_contour[0], (255))

#-- Smooth mask, then blur it
mask = cv2.dilate(mask, None, iterations=MASK_DILATE_ITER)
mask = cv2.erode(mask, None, iterations=MASK_ERODE_ITER)
mask = cv2.GaussianBlur(mask, (BLUR, BLUR), 0)
mask_stack = np.dstack([mask]*3)    # Create 3-channel alpha mask

#-- Blend masked img into MASK_COLOR background
mask_stack  = mask_stack.astype('float32') / 255.0         
img         = img.astype('float32') / 255.0    
masked = (mask_stack * img) + ((1-mask_stack) * MASK_COLOR)  
masked = (masked * 255).astype('uint8')                    

cv2.imshow('img', masked)                                   # Display
cv2.waitKey()
cv2.imwrite("WTF.jpg",masked)

But this code only works for only this image

What should be changed in the code to make it to work for different images

Answer 1

Local Optimal Solution

# Original Code
CANNY_THRESH_2 = 200

# Change to
CANNY_THRESH_2 = 100

####### Change below worth to try but not necessary

# Original Code
mask = np.zeros(edges.shape)
cv2.fillConvexPoly(mask, max_contour[0], (255))

# Change to
for c in contour_info:
    cv2.fillConvexPoly(mask, c[0], (255))

Effects

• Test Image
  • Similar color of background, hair and skin

• Original Output
  • original output

• original edges

• Apply all contour rather than max contour with same edge threshold

  • slightly better

  

• Canny Thresh 2 set as 100, apply all contour

  • much better

• stronger edges

• Canny Thresh 2 set as 40, apply all contour
  • edges starts to become not so sharp

Reasoning

1. Program Behavior

   The program searches edges and builds contours. Get the max contour and recognize as human face. Then apply mask.

2. Problem

   Not easy to deal with similar color between background and human face. Blond hair and skin color makes it's hard to find correct edges with the original threshold.

   Max contour means when images have strong and big vertex like the scarf in test image, it's easy to lose track of some area. But it really depends on what kind of image it is after your human face recognition process.