Driving a Windows GUI program from a script

Question

I have to use a Windows simulation package to perform a repetitive task with slightly different options each time.

Since I hate repetitive clicking, on grounds of both laziness and the amount of errors that a human introduces, I would like to drive this program automatically. The program in question doesn't support scripting, there is no API, no COM, nada, nyet, nravin. As far as I can tell, the only way to drive this program automatically is to imitate a human (i.e. keyboard and mouse macros.)

I am aware of AutoHotKey but I don't think it does what I want. (Or it might do what I want, but its scripting language is horrible.)

The requirements are:

• Must allow time delays between actions, or event detection to trigger actions.

  The simulations can take up to ten minutes to run, so the GUI driver would have to wait until the simulation finishes before starting a new one.

  One way to do this would be to just wait ten minutes and hope that the simulation has finished. An alternative way is to make it event-driven, i.e. watch for the "Simulation running..." dialog to disappear and be replaced by a "Simulation complete" dialog.

• Must allow composition of complex keyboard input.

  Some of the keyboard input required is different for each simulation run. For example the simulation description might take the format [Project name][Scenario name][Option 1][Option 2]... and this would have to be entered for each simulation.

  I am aware that AutoHotKey allows a basic level of input customisation, but my casual reading of the documentation makes the scripting language look like some kind of eldritch horror.

• This is for work, so any solution must be free for commercial use.

I will accept any solution that fits the criteria above, but I have a strong preference for something I can drive from Python. However I would also accept automated GUI-testing tools that I could customise to do what I want - possibly a Win32 GUI equivalent of Selenium for browsers? - keyboard macro recorders that will generate custom output, or anything else that works.

Answer 1

Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). Sikuli includes Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API's support. You can programmatically control a web page, a Windows/Linux/Mac OS X desktop application, or even an iphone or android application running in a simulator or via VNC.

Look at Sikuli, it worked for me.

Answer 2

Take a look at Automa - it is written in Python. It can be used either as a standalone tool or as a Python library in your own scripts:

from automa.api import *

It allows automation of any Windows application through commands like click, press, write, etc.

Some examples of the automation scripts can be found at http://www.getautoma.com/blog/category/ui-automation-examples

Disclaimer: I'm one of Automa's developers.

Answer 3

Look at this https://pywinauto.github.io/

You can use python script itself to control your windows application.

Advantage is:

• no need to learn new language/syntax
• integrates easily with other existing script

Answer 4

Give Autohotkey another look, from you requirements it seems fit for the job.

Alternatively check UI Automation from Microsoft: http://msdn.microsoft.com/en-us/library/ms747327.aspx and also white: http://white.codeplex.com/

Answer 5

You can use PyAutoGUI library for Python which works on Windows, macOS, and Linux.

Must allow time delays between actions.

Example to type with quarter-second pause in between each key:

import pyautogui
pyautogui.typewrite('Hello world!', interval=0.25)

Here is the example to set up a 2.5-second pause after each PyAutoGUI call:

pyautogui.PAUSE = 2.5

Must allow composition of complex keyboard input.

Checkout keyboard control functions where you can use pyautogui.typewrite to type something out. You can pass variables to allow a complex keyboard input.

Event detection to trigger actions.

You can use locate functions to visually find something on the screen and make the condition based on that within a simple loop.

Solution must be free for commercial use.

It is licensed under the BSD which allows commercial use.

---

See also:

• Which is the easiest way to simulate keyboard and mouse on Python?
• Python GUI automation library for simulating user interaction in apps.