How to crop multiple rectangles or squares from JPEG?

Question

I have a jpeg from where I want to crop a portion containing graph (the one in the bottom portion).

As of now I used this code to achieve the same:

from PIL import Image

img = Image.open(r'D:\aakash\graph2.jpg')
area = (20, 320, 1040, 590)
img2 = img.crop(area)
# img.show()
img2.show()

But I achieved this by guessing the x1, y1, x2, y2 multiple times to arrive at this (guess work).

Image before cropping:

Image after cropping:

I'm totally novice in image cropping based on some logic. How can I successfully crop all graphs to create separate images given the positions are same?

Update: I believe, it is not a possible duplicate of that problem because, even though logically that's same, but the way the clustering logic will work is different. In that question, there are only 2 vertical white lines to divide, but here, there's two horizontal and two vertical lines, and I hardly have a clue on how to use KMeans to solve this kind of image clustering.

Help from someone who's an expert with sklearn's KMeans to solve this kind of problem shall be highly appreciated.

Answer 1

Here's a way to do it, using OpenCV findContours() method.

#!/usr/bin/env python3

import numpy as np
import cv2

# Load image
im = cv2.imread('article.jpg',cv2.IMREAD_UNCHANGED)

# Create greyscale version
gr = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)

# Threshold to get black and white
_,grthresh = cv2.threshold(gr,230,255,cv2.THRESH_BINARY)
cv2.imwrite('result-1.png',grthresh)

# Median filter to remove JPEG noise
grthresh = cv2.medianBlur(grthresh,11)
cv2.imwrite('result-2.png',grthresh)

# Find contours
im2, contours, hierarchy = cv2.findContours(grthresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE)

# Look through contours, checking what we found
blob = 0
for i in range(len(contours)):
    area  = cv2.contourArea(contours[i])
    # Only consider ones taller than around 100 pixels and wider than about 300 pixels
    if area > 30000:
        # Get cropping box and crop
        rc = cv2.minAreaRect(contours[i])
        box = cv2.boxPoints(rc)
        Xs = [ box[0,0], box[1,0], box[2,0], box[3,0]]
        Ys = [ box[0,1], box[1,1], box[2,1], box[3,1]]
        x0 = int(round(min(Xs)))
        x1 = int(round(max(Xs)))
        y0 = int(round(min(Ys)))
        y1 = int(round(max(Ys)))
        cv2.imwrite(f'blob-{blob}.png', im[y0:y1,x0:x1])
        blob += 1

It gives you these files:

-rw-r--r--@ 1 mark  staff  248686  6 Jun 09:00 blob-0.png
-rw-r--r--@ 1 mark  staff   92451  6 Jun 09:00 blob-1.png
-rw-r--r--@ 1 mark  staff  101954  6 Jun 09:00 blob-2.png
-rw-r--r--@ 1 mark  staff  102373  6 Jun 09:00 blob-3.png
-rw-r--r--@ 1 mark  staff  633624  6 Jun 09:00 blob-4.png

The intermediate, debug files are result-1.png:

And result-2.png: