How to use tweening in Python, without losing accuracy?

Question

I've been struggling to use tweening to make mouse movements smooth in Python, I am currently trying to automate some repetitive tasks.

I've tried to use tweening to remove some of the roughness that occurs without smoothing applied, however by doing so I am losing a noticeable amount of accuracy, after all my dy and dx values are getting split by a number I end up with remainders. This could possibly be solved by getting the greatest common factor on both my values (since both dx and dy need to be split by the same number) unfortunately this leads to a too small of a GCD.

Since the mouse cannot move the remainder of a pixel on a screen I end up a with noticeable loss of accuracy.

Question: How to apply tweening on mouse movements, without losing accuracy?

import pytweening
import win32api
import win32con
from time import sleep

dy = [50, 46, 42, 38, 33, 29, 24, 20, 15, 10, 10]
dx = [-35, 6, -55, -43, 0, 17, 29, 38, 42, 42, 38]

while True:

    count = 0

    values = [(pytweening.getPointOnLine(0, 0, x, y, 0.20)) for x, y in zip(dx, dy)]

    while win32api.GetAsyncKeyState(win32con.VK_RBUTTON) and win32api.GetAsyncKeyState(win32con.VK_LBUTTON):

        if count < len(dx):

            for _ in range(5):
                win32api.mouse_event(1, int(values[count][0]), int(values[count][1]), 0, 0)
                sleep(0.134 / 5)

            count += 1

Answer 1

The fundamental problem here is that you are using relative movement in integer amounts, which will not add up to the total movement you are looking for. If you only want to move linearly, you also don't need PyTweening at all. How about this solution?

import win32api
import win32con
from time import sleep

Npoints = 5
sleeptime = 0.134 / Npoints

dys = [50, 46, 42, 38, 33, 29, 24, 20, 15, 10, 10]
dxs = [-35, 6, -55, -43, 0, 17, 29, 38, 42, 42, 38]

x, y = win32api.GetCursorPos()

for dx, dy in zip(dxs, dys):
    ddx = dx/Npoints
    ddy = dy/Npoints
    for _ in range(Npoints):
        x += ddx
        y += ddy

        win32api.SetCursorPos(int(x), int(y))
        sleep(sleeptime)

Note that there will still be some very small round-off error and that the cursor will move in a straight line between the points. If the cursor starts at (0, 0), this is the shape it will make (the red crosses are the points where the cursor will be set to):

If you wanted to move in smooth curves through the points and you're OK with using numpy and scipy, this will handle that:

import numpy as np
import scipy.interpolate as sci

totalpoints = 50  # you can set this to a larger number to get closer spaced points
x, y = win32api.GetCursorPos()

# work out absolute coordinates of new points
xs = np.cumsum([x, *dxs])
ys = np.cumsum([y, *dys])

# fit spline between the points (s=0 makes the spline hit all the points)
tck, u = sci.splprep([xs, ys], s=0)

# Evaluate the spline and move to those points
for x, y in zip(*sci.splev(np.linspace(0, 1, totalpoints), tck)):
    win32api.SetCursorPos(int(x), int(y))
    sleep(sleeptime)

This results in positions as shown below:

Answer 2

Question: Tweening, without losing accuracy?

Reference:

• PyTweening - getLinePoint()

  x, y = getLinePoint(startPoint x, startPoint y, endPoint x, endPoint y, intervall)

  The getLinePoint() function finds a point on the provided line.

---

1. Cast your lists, dx anddy, into a list of tuple(x, y)

   dx = [-35, 6, -55, -43, 0, 17, 29, 38, 42, 42, 38]
   dy = [50, 46, 42, 38, 33, 29, 24, 20, 15, 10, 10]
   
   points = list(zip(dx, dy))
   print(points)
   

   Output:

   [(-35, 50), (6, 46), (-55, 42), (-43, 38), (0, 33), (17, 29), (29, 24), (38, 20), (42, 15), (42, 10), (38, 10)]
   

2. Process this list of points in a double for loop.

   import pytweening
   
   for startPoint in points:
       for endPoint in points:
           x, y = pytweening.getPointOnLine(startPoint[0], startPoint[1],
                                            endPoint[0], endPoint[1], 
                                            0.20)
           x, y = int(x), int(y)
           print('{}, '.format((x, y)), end='')
   
           # win32api.mouse_event(1, x, y, 0, 0)
           # sleep(0.134)
   

   Output: The End Points are allways reached!

   First move from (-35, 50) to (6, 46):
   (-35, 50), (-26, 49), (-39, 48), (-36, 47), (-28, 46), (-24, 45),(-22, 44),
   (-20, 44), (-19, 43), (-19, 42), (-20, 42), (-2, 46), (6, 46)
   
   ... (omitted for brevity)
   
   Last move from (42, 10) to (38, 10):  
   (42, 10), (41, 10), (23, 18), (31, 17), (19, 16), (21, 15), (30, 14),
   (33, 13), (36, 12), (38, 12), (38, 11), (38, 10), (38, 10)
   

---

Tested with Python: 3.6 - pytweening: 1.0.3