Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Robustly crop rotated bounding box on photos

I'm trying to extract the rotated bounding box of contours robustly. I would like to take an image, find the largest contour, get its rotated bounding box, rotate the image to make the bounding box vertical, and crop to size.

For a demonstration, here is an original image linked in the following code. I would like to end up with that shoe rotated to vertical and cropped to size. The following code from this answer seems to work on simple images like opencv lines, etc., but not on photos.

enter image description here

Ends up with this, which is rotated and cropped wrong:

enter image description here

EDIT: After changing the threshold type to cv2.THRESH_BINARY_INV, it now is rotated correctly but cropped wrong:

enter image description here

import cv2
import matplotlib.pyplot as plt
import numpy as np
import urllib.request
plot = lambda x: plt.imshow(x, cmap='gray').figure


url = 'https://i.imgur.com/4E8ILuI.jpg'
img_path = 'shoe.jpg'

urllib.request.urlretrieve(url, img_path)
img = cv2.imread(img_path, 0)
plot(img)


threshold_value, thresholded_img = cv2.threshold(
    img, 250, 255, cv2.THRESH_BINARY)
_, contours, _ = cv2.findContours(thresholded_img, 1, 1)
contours.sort(key=cv2.contourArea, reverse=True)

shoe_contour = contours[0][:, 0, :]
min_area_rect = cv2.minAreaRect(shoe_contour)

def crop_minAreaRect(img, rect):

    # rotate img
    angle = rect[2]
    rows, cols = img.shape[0], img.shape[1]
    M = cv2.getRotationMatrix2D((cols / 2, rows / 2), angle, 1)
    img_rot = cv2.warpAffine(img, M, (cols, rows))

    # rotate bounding box
    rect0 = (rect[0], rect[1], 0.0)
    box = cv2.boxPoints(rect)
    pts = np.int0(cv2.transform(np.array([box]), M))[0]
    pts[pts < 0] = 0

    # crop
    img_crop = img_rot[pts[1][1]:pts[0][1],
                       pts[1][0]:pts[2][0]]

    return img_crop


cropped = crop_minAreaRect(thresholded_img, min_area_rect)
plot(cropped)

How can I get the correct cropping?


like image 954
Hatshepsut Avatar asked Jul 24 '17 18:07

Hatshepsut


People also ask

How do you crop a rotated bounding box?

In the above code, we first find the rectangle enclosing the text area based on the four points we provide using the cv2. minAreaRect() method. Then in function crop_rect() , we calculate a rotation matrix and rotate the original image around the rectangle center to straighten the rotated rectangle.

Can bounding boxes be rotated?

Bounding boxes are constructed by first creating an axis-aligned box (left), and then rotating by theta (right). Many datasets (for example, COCO and ISPRS) come with segmentation masks. These masks can be converted into rotated bounding boxes by using a geometry package.


1 Answers

After some research, this is what I get:

enter image description here

This is how I get it:

  • pad the original image on each side (500 pixels in my case)
  • find the four corner points of the shoe (the four points should form a polygon enclosing the shoe, but do not need to be exact rectangle)
  • employing the code here to crop the shoe:

img = cv2.imread("padded_shoe.jpg")
# four corner points for padded shoe
cnt = np.array([
    [[313, 794]],
    [[727, 384]],
    [[1604, 1022]],
    [[1304, 1444]]
])
print("shape of cnt: {}".format(cnt.shape))
rect = cv2.minAreaRect(cnt)
print("rect: {}".format(rect))

box = cv2.boxPoints(rect)
box = np.int0(box)
width = int(rect[1][0])
height = int(rect[1][1])

src_pts = box.astype("float32")
dst_pts = np.array([[0, height-1],
                    [0, 0],
                    [width-1, 0],
                    [width-1, height-1]], dtype="float32")
M = cv2.getPerspectiveTransform(src_pts, dst_pts)
warped = cv2.warpPerspective(img, M, (width, height))

Cheers, hope it helps.

like image 86
jdhao Avatar answered Nov 15 '22 00:11

jdhao