How to record mouse and keyboard movement simultaneously with Python?

Question

I want to create a function that records both mouse and keyboard events until a specific key is pressed and then replays them together.

I think that this can be achieved with the keyboard and mouse modules. In an earlier question, I asked how to record the mouse movement until a key is pressed, and I got the following code:

import mouse
import keyboard

events = []                 #This is the list where all the events will be stored
mouse.hook(events.append)   #starting the mouse recording
keyboard.wait("a")          #Waiting for 'a' to be pressed
mouse.unhook(events.append) #Stopping the mouse recording
mouse.play(events)          #Playing the recorded events

That works fine. As both modules were made by the same people, I assumed that the same would work with the keyboard module. But it doesn't.

mouse_events = []
keyboard_events = []

mouse.hook(mouse_events.append)
keyboard.hook(keyboard_events.append)

keyboard.wait("a")

mouse.unhook(events.append)
keyboard.unhook(events.append)
keyboard.play(events)

The keyboard.hook(events.append) line in the code above throws an error: TypeError: unhashable type: 'list'.

I tried to check the module files but I fail to understand most of it.

So, to summarize: How can I start the mouse and keyboard recording at the same moment, stop them both at the same time and run both simultaneously? Are the mouse and keyboard modules the best option to achieve this?

Answer 1

There is a problem with your code.
Your lists are:

mouse_events = []
keyboard_events = []

But you are using events.append rather than the list name. Looks like you forgot to modify the code.

---

The error is thrown because the module keyboard uses dict for hook unlike module mouse and you can't use list as keys.

You can solve this by using lambda:

keyboard.hook(lambda _: keyboard_events.append(_))

---

There is more simpler way to do this without using hook but its only for module keyboard

Use start_recording() and stop_recording()

1) start_recording() to enable the recording of keyboard events. It doesn't take a callback and you can record once at a time.
2) stop_recording() to stop the started recording. It returns the list of recorded events.

_{mouse module doesn't have stop/start_recording}
So your final code will look like this:

import mouse
import keyboard

mouse_events = []


mouse.hook(mouse_events.append)
keyboard.start_recording()       #Starting the recording

keyboard.wait("a")

mouse.unhook(mouse_events.append)
keyboard_events = keyboard.stop_recording()  #Stopping the recording. Returns list of events

---

Playing the events together:

The only way to play both events at the same time is to use threading

Here is an example for your code:

import threading
import mouse
import keyboard

mouse_events = []


mouse.hook(mouse_events.append)
keyboard.start_recording()

keyboard.wait("a")

mouse.unhook(mouse_events.append)
keyboard_events = keyboard.stop_recording()

#Keyboard threadings:

k_thread = threading.Thread(target = lambda :keyboard.play(keyboard_events))
k_thread.start()

#Mouse threadings:

m_thread = threading.Thread(target = lambda :mouse.play(mouse_events))
m_thread.start()

#waiting for both threadings to be completed

k_thread.join() 
m_thread.join()