Python: Use mouse to draw a rectangle around objects in any window? Also store start and end coordinates as variables relative to said window?

Question

I'm very new to Python and need an approach for accomplishing an important function I've done before in macro-scripting languages:

I have a Python program that will be processing a screengrab image using pyscreenshot. In order to feed pyscreenshot the necessary x1, y1, x2, y2 coordinates to create the image, I need the user to define the portion of the screen by drawing a rectangle around it. Think "snipping" tool in Windows and cmd+shift+4 in OSX.

I am open to using any BSD module necessary but I need module support for OSX and Windows.

Every example I've found is for how to draw a rectangle within a canvas or on an existing image file. These examples don't help me because I need the coordinates to be drawn-over and relative to any OS window the user chooses.

Thanks so much for the help!!!

Looked at tutorials to draw rectangles in opencv, pyautogui, tkinter but all drew the rectangles over a window or canvas drawn by the program, not the OS.

Answer 1

I would first grab a screenshot of your current desktop, display it in a Canvas, and let your user to draw a rectangle on it. Then retrieve the bounding box of the rectangle and perform an actual grab on the desktop. Below is a basic sample (drag to create an image):

import tkinter as tk
from PIL import ImageGrab, ImageTk

class GUI(tk.Tk):
    def __init__(self):
        super().__init__()
        self.withdraw()
        self.attributes('-fullscreen', True)

        self.canvas = tk.Canvas(self)
        self.canvas.pack(fill="both",expand=True)

        image = ImageGrab.grab()
        self.image = ImageTk.PhotoImage(image)
        self.photo = self.canvas.create_image(0,0,image=self.image,anchor="nw")

        self.x, self.y = 0, 0
        self.rect, self.start_x, self.start_y = None, None, None
        self.deiconify()

        self.canvas.tag_bind(self.photo,"<ButtonPress-1>", self.on_button_press)
        self.canvas.tag_bind(self.photo,"<B1-Motion>", self.on_move_press)
        self.canvas.tag_bind(self.photo,"<ButtonRelease-1>", self.on_button_release)

    def on_button_press(self, event):
        self.start_x = event.x
        self.start_y = event.y
        self.rect = self.canvas.create_rectangle(self.x, self.y, 1, 1, outline='red')

    def on_move_press(self, event):
        curX, curY = (event.x, event.y)
        self.canvas.coords(self.rect, self.start_x, self.start_y, curX, curY)

    def on_button_release(self, event):
        bbox = self.canvas.bbox(self.rect)
        self.withdraw()
        self.new_image = ImageTk.PhotoImage(ImageGrab.grab(bbox))
        self.attributes('-fullscreen', False)
        self.title("Image grabbed")
        self.canvas.destroy()
        self.deiconify()
        tk.Label(self,image=self.new_image).pack()

root = GUI()

root.mainloop()