Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Algorithm to emulate mouse movement as a human does?

I need to test a software that treats some mouse movements as "gestures". For such a task I need to emulate mouse movement from point A to point B, not in straight line, but as a real mouse moves - with curves, a bit of jaggedyness etc. Is there any available solution (algorithm/code itself, not a library/exe) that I can use?

Of course I can write some simple sinusoidal math by myself, but this would be a very crude emulation of a human hand leading a mouse. Perhaps such a task has been solved already numerous times, and I can just borrow an existing code? :)

like image 390
grigoryvp Avatar asked Jan 09 '10 17:01

grigoryvp


People also ask

What are the mouse movements?

Mouse movements can be used to infer a user's intent and focus while browsing a website. By using mouse movements in usability testing, researchers can determine if users are confused, if their expectations are met, where their attention is focused, and much additional information.

How do you move the cursor in Python?

This code uses moveTo() function, which takes x and y coordinates, and an optional duration argument. This function moves your mouse pointer from it's current location to x, y coordinate, and takes time as specified by duration argument to do so.


4 Answers

For anyone in the future: I developed a library for Java, that does exactly what OP is asking. The noise/jaggedness in movement, sinusoidal arcs, overshooting the position a bit, etc. Plus the library is written with extension and configuration possibilities in mind, so anyone can fine tune it, if the default solution is not matching the case. Available from Maven Central now.

https://github.com/JoonasVali/NaturalMouseMotion

like image 124
Joonas Vali Avatar answered Oct 21 '22 22:10

Joonas Vali


Take a look at the Mouse.simba file which is part of the SRL Framework, a macro program designed to work with online games like Runescape. Runescape has extensive macro detection capabilities so all parts of the SRL Framework have been developed to look as human as possible - the mouse functions in particular.

The code is in Pascal but should be quite easy to read. Look at the MMouse() procedure by BenLand100 it moves very realistically doing advanced movements like loops and overshooting its target as well as continuously varying accelerations and directions (also now has laptop touchpad type movements). If you want to test it you'll need to download SIMBA and enable the SRL-include.

like image 29
Callum Rogers Avatar answered Oct 21 '22 23:10

Callum Rogers


How about recording some real gestures and making a way to play them back? That's going to be as real as anything you can synthesize, and it would be repeatable (which is nice for testing).

like image 5
Adrian McCarthy Avatar answered Oct 21 '22 23:10

Adrian McCarthy


Create a simple test app which shows a random positioned dot every two seconds. Follow the dot yourself and record your natural mouse movements.

like image 2
Viktor Sehr Avatar answered Oct 21 '22 22:10

Viktor Sehr