Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a way to send a click event to a window in the background in python?

So I'm trying to build a bot to automate some actions in a mobile game that I'm running on my pc through Bluestacks.

My program takes a screenshot of the window, looks for certain button templates in the image and returns their coordinates.

I would now like to be able to send a click event to the window at those coordinates, but since I would also like to do other things while the bot runs in the background I'm looking for a way to send the mouse event directly to the window (even if it's minimized/in the background), without influencing the movement of the mouse while I'm doing other stuff or bringing the window to the foreground/unminimizing it. Is this possible?

like image 664
frikai Avatar asked Mar 04 '23 01:03

frikai


2 Answers

Based on Dmitry's answer. The x and y must be the coordinates relative to the Bluestacks window not the screen.

def click(x, y):
    hWnd = win32gui.FindWindow(None, "BlueStacks")
    lParam = win32api.MAKELONG(x, y)

    hWnd1= win32gui.FindWindowEx(hWnd, None, None, None)
    win32gui.SendMessage(hWnd1, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, lParam)
    win32gui.SendMessage(hWnd1, win32con.WM_LBUTTONUP, None, lParam)

click(100,100)
like image 95
Onil González Avatar answered Mar 05 '23 16:03

Onil González


I solved this problem on windws 10 using win32api.

In Spy++ I looked at the mouse messages that occur when I click in Bluestacks. I found that I should find the hwnd of bluestacks child window with the title "BlueStacks Android PluginAndroid". And send them mouse click events:

lParam = win32api.MAKELONG(x, y)
win32api.PostMessage(hWnd, win32con.WM_LBUTTONDOWN, win32con.MK_LBUTTON, lParam)
win32api.PostMessage(hWnd, win32con.WM_LBUTTONUP, None, lParam)

This works for me even if the window is minimized to the tray.

like image 23
Dmitry Modenov Avatar answered Mar 05 '23 15:03

Dmitry Modenov