Pyautogui: Mouse Movement with bezier curve

Question

I'm trying to move the mouse in a bezier curve motion in Pyautogui to simulate more of a human movement as seen here:

There is some tweening / easing functions within pyautogui but none of which represent a bezier curve type movement. I created a small script to calculate the random places it will hit before ultimately hitting its destination.

Default "Robot" linear path:

Unfortunately, which each destination the mouse temporarily stops.

import pyautogui
import time
import random
print "Randomized Mouse Started."
destx = 444;
desty = 631;
x, y = pyautogui.position() # Current Position
moves = random.randint(2,4)
pixelsx = destx-x
pixelsy = desty-y
if moves >= 4:
        moves = random.randint(2,4)
avgpixelsx = pixelsx/moves
avgpixelsy = pixelsy/moves
print "Pixels to be moved X: ", pixelsx," Y: ",pixelsy, "Number of mouse movements: ", moves, "Avg Move X: ", avgpixelsx, " Y: ", avgpixelsy

while moves > 0:
        offsetx = (avgpixelsx+random.randint(-8, random.randint(5,10)));
        offsety = (avgpixelsy+random.randint(-8, random.randint(5,10)));
        print x + offsetx, y + offsety, moves
        pyautogui.moveTo(x + offsetx, y + offsety, duration=0.2)
        moves = moves-1
        avgpixelsx = pixelsx / moves
        avgpixelsy = pixelsy / moves

Info:

• Windows 10
• Python 2.7
• Willing to use other libraries, Python version if necessary

I've seen this post: python random mouse movements

but can't figure out how to define a "start and stop" position. The answer is pretty close to what I'm looking for.

Any ideas on how to accomplish this?

Answer 1

Using scipy, numpy and anything that can simply move mouse cursor:

import pyautogui
import random
import numpy as np
import time
from scipy import interpolate
import math

def point_dist(x1,y1,x2,y2):
    return math.sqrt((x2 - x1) ** 2 + (y2 - y1) ** 2)

cp = random.randint(3, 5)  # Number of control points. Must be at least 2.
x1, y1 = pyautogui.position()  # Starting position

# Distribute control points between start and destination evenly.
x = np.linspace(x1, x2, num=cp, dtype='int')
y = np.linspace(y1, y2, num=cp, dtype='int')

# Randomise inner points a bit (+-RND at most).
RND = 10
xr = [random.randint(-RND, RND) for k in range(cp)]
yr = [random.randint(-RND, RND) for k in range(cp)]
xr[0] = yr[0] = xr[-1] = yr[-1] = 0
x += xr
y += yr

# Approximate using Bezier spline.
degree = 3 if cp > 3 else cp - 1  # Degree of b-spline. 3 is recommended.
                                  # Must be less than number of control points.
tck, u = interpolate.splprep([x, y], k=degree)
# Move upto a certain number of points
u = np.linspace(0, 1, num=2+int(point_dist(x1,y1,x2,y2)/50.0))
points = interpolate.splev(u, tck)

# Move mouse.
duration = 0.1
timeout = duration / len(points[0])
point_list=zip(*(i.astype(int) for i in points))
for point in point_list:
    pyautogui.moveTo(*point)
    time.sleep(timeout)

And you can remove any built-in delay in pyautogui by setting:

# Any duration less than this is rounded to 0.0 to instantly move the mouse.
pyautogui.MINIMUM_DURATION = 0  # Default: 0.1
# Minimal number of seconds to sleep between mouse moves.
pyautogui.MINIMUM_SLEEP = 0  # Default: 0.05
# The number of seconds to pause after EVERY public function call.
pyautogui.PAUSE = 0  # Default: 0.1

P.S.: Example above doesn't require any of those settings as it doesnt use public moveTo method.

Answer 2

For a simple solution, you can try using numpy with the bezier library:

import pyautogui
import bezier
import numpy as np


# Disable pyautogui pauses (from DJV's answer)
pyautogui.MINIMUM_DURATION = 0
pyautogui.MINIMUM_SLEEP = 0
pyautogui.PAUSE = 0

# We'll wait 5 seconds to prepare the starting position
start_delay = 5 
print("Drawing curve from mouse in {} seconds.".format(start_delay))
pyautogui.sleep(start_delay)

# For this example we'll use four control points, including start and end coordinates
start = pyautogui.position()
end = start[0]+600, start[1]+200
# Two intermediate control points that may be adjusted to modify the curve.
control1 = start[0]+125, start[1]+100
control2 = start[0]+375, start[1]+50

# Format points to use with bezier
control_points = np.array([start, control1, control2, end])
points = np.array([control_points[:,0], control_points[:,1]]) # Split x and y coordinates

# You can set the degree of the curve here, should be less than # of control points
degree = 3
# Create the bezier curve
curve = bezier.Curve(points, degree)
# You can also create it with using Curve.from_nodes(), which sets degree to len(control_points)-1
# curve = bezier.Curve.from_nodes(points)

curve_steps = 50  # How many points the curve should be split into. Each is a separate pyautogui.moveTo() execution
delay = 1/curve_steps  # Time between movements. 1/curve_steps = 1 second for entire curve

# Move the mouse
for i in range(1, curve_steps+1):
    # The evaluate method takes a float from [0.0, 1.0] and returns the coordinates at that point in the curve
    # Another way of thinking about it is that i/steps gets the coordinates at (100*i/steps) percent into the curve
    x, y = curve.evaluate(i/curve_steps)
    pyautogui.moveTo(x, y)  # Move to point in curve
    pyautogui.sleep(delay)  # Wait delay

I came up with this trying to write something to draw SVG Paths with the mouse. Running the above code will make your mouse move along the same path as below. The red dots are positioned at each of the control points that define the curve.

Note that you'll have to add pyautogui.mouseDown() before and pyautogui.mouseUp() after the loop at the end of the script if you want to click and drag like I did here in GIMP:

You can check out the bezier docs here: https://bezier.readthedocs.io/en/stable/index.html

Answer 3

you just need know is the move_mouse((300,300))will let you mouse arrive (300,300),then never change.look at the implement,it just call the WIN32 api mouse_event.read something about it,you will find there are no "start and stop" position.i don't know how to draw bezier curve.

    while True:
        pos = (random.randrange(*x_bound),random.randrange(*y_bound))
        move_mouse(pos)
        time.sleep(1.0/steps_per_second)

look,that is the secret of animation.all you need do is write a pos = draw_bezier_curve(t)