How to simulate keys in game with pywinauto

Question

I've been trying various things for a year. I am a at beginner's level in python. Did the first two questions in Project Euler.

I have tried a few methods to try and simulate keys in games I play. I can easily do this with autohotkey and macro keyboards/mouse. However, I would like to accomplish this through Python or C.

My guess is that win32 api is ignored in video games and I need to simulate key presses through Direct X.

Thank you in advance. Here is my latest attempt... it failed.

I do have to grab/change the handle every time I run a new instance of the game.

My simulated keys would work in the browser and notepad, just not in game. By not working, I mean there is no user input.

The following code would switch to the window but will not simulate an input from the user.

import pywinauto
import time
from pywinauto import application
app = application.Application()
app.connect_(handle = 0x14002a)
dialogs = app.windows_(handle = 0x14002a)
dlg = app.top_window_()
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")
time.sleep(1)
app.MapleStory.TypeKeys("%A")

Answer 1

We have had a long journey my past self. Though we have much to learn here is what we discovered:

Sending Virtual Keys will be ignored if the game is running on DirectX. Use sendinput and send the scan_codes instead.

# http://www.gamespp.com/directx/directInputKeyboardScanCodes.html

import ctypes
import time

SendInput = ctypes.windll.user32.SendInput

W = 0x11
A = 0x1E
S = 0x1F
D = 0x20
Z = 0x2C
UP = 0xC8
DOWN = 0xD0
LEFT = 0xCB
RIGHT = 0xCD
ENTER = 0x1C 

# C struct redefinitions 
PUL = ctypes.POINTER(ctypes.c_ulong)
class KeyBdInput(ctypes.Structure):
    _fields_ = [("wVk", ctypes.c_ushort),
                ("wScan", ctypes.c_ushort),
                ("dwFlags", ctypes.c_ulong),
                ("time", ctypes.c_ulong),
                ("dwExtraInfo", PUL)]

class HardwareInput(ctypes.Structure):
    _fields_ = [("uMsg", ctypes.c_ulong),
                ("wParamL", ctypes.c_short),
                ("wParamH", ctypes.c_ushort)]

class MouseInput(ctypes.Structure):
    _fields_ = [("dx", ctypes.c_long),
                ("dy", ctypes.c_long),
                ("mouseData", ctypes.c_ulong),
                ("dwFlags", ctypes.c_ulong),
                ("time",ctypes.c_ulong),
                ("dwExtraInfo", PUL)]

class Input_I(ctypes.Union):
    _fields_ = [("ki", KeyBdInput),
                 ("mi", MouseInput),
                 ("hi", HardwareInput)]

class Input(ctypes.Structure):
    _fields_ = [("type", ctypes.c_ulong),
                ("ii", Input_I)]

# Actuals Functions

def pressKey(hexKeyCode):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    ii_.ki = KeyBdInput( 0, hexKeyCode, 0x0008, 0, ctypes.pointer(extra) )
    x = Input( ctypes.c_ulong(1), ii_ )
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))

def releaseKey(hexKeyCode):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    ii_.ki = KeyBdInput( 0, hexKeyCode, 0x0008 | 0x0002, 0, 
ctypes.pointer(extra) )
    x = Input( ctypes.c_ulong(1), ii_ )
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))

if __name__ == '__main__':
    pressKey(0x11)
    time.sleep(1)
    releaseKey(0x11)
    time.sleep(1)

Source: Simulate Python keypresses for controlling a game http://www.gamespp.com/directx/directInputKeyboardScanCodes.html

Thank you Sentdex for awesome tutorials on Machine Learning. I've been wanting to do this to some of my favorite games but couldn't get the keys across (due to DirectX).

Next step: Windows Driver Kit to emulate keypresses...

Answer 2

I don't know if this is too late, but yes for some games you can simulate mouse / keyboard events by using scan codes instead of virtual keys with SendInput

However as for Maplestory specifically, it doesn't respond to both virtual keys and scan codes.

The only possible way to simulate keyboard events within this game that I know of is to use kernel drivers. One example is to use WinIo to write directly to i8042 and i8048 controllers. The command 0xD2 on 8042 is used specifically to simulate hardware level inputs. However, WinIo opens up a big security hole by exposing hardware ports directly to user space programs, so I couldn't say I recommend it.