MySQL waiting for handler commit

Question

After upgrade from MySQL 5.7 to 8.0, I have once in a while queries stuck forever in waiting for handler commit state.

42425 generic %:45408 objects_db Killed 59608 waiting for handler commit COMMIT
42426 generic %:45410 objects_db Killed 59608 waiting for handler commit commit
42434 generic %:45428 objects_db Killed 59608 waiting for handler commit COMMIT
42444 generic %:45454 objects_db Killed 59608 waiting for handler commit REPLACE INTO vvObjectSearch(CompanyID, ObjectTypeID, ObjectID, PropertyID, `Value`)

1. There is no way to kill these queries
2. There is no way to restart MySQL service - it hangs forever, the only way is kill -9
3. Tried to change range_optimizer_max_mem_size to high values including 0
4. Nothing in error log

The server is in master-master replication with another server. Both servers are exactly the same, but serve different clients. The other server for some reason does not have this issue. After several weeks of research, I rebuilt the server from scratch and restored a fresh backup (xtrabackup) from the second server, restoring replication. Didn't help. Any ideas?

Answer 1

After fighting for 3 months, I will answer my own question in case someone is having a similar problem. Background: In MySQL 8.0.27 the default of replica_parallel_workers has changed from zero (single thread) to 4. I changed replica_parallel_workers to zero and the problem has gone. Looks like a bug in MySQL causing some deadlock between the slave threads.