Case of the confounding key press caper

Question

Background

Developing a rudimentary, open-source keyboard and mouse on-screen display desktop application for screen casting, called KmCaster:

The application uses the JNativeHook library to receive global keyboard and mouse events, because Swing's Key and Mouse listeners are restricted to receiving events directed at the application itself.

Problem

When the application loses focus, the user interface shows intermittent key presses, rather than every key press. Yet the console shows that the application has received every key press.

Code

A short, self-contained, compileable example:

import org.jnativehook.GlobalScreen;
import org.jnativehook.NativeHookException;
import org.jnativehook.keyboard.NativeKeyEvent;
import org.jnativehook.keyboard.NativeKeyListener;

import javax.swing.*;

import static java.util.logging.Level.OFF;
import static java.util.logging.Logger.getLogger;
import static javax.swing.SwingUtilities.invokeLater;
import static org.jnativehook.GlobalScreen.*;
import static org.jnativehook.keyboard.NativeKeyEvent.getKeyText;

public class Harness extends JFrame implements NativeKeyListener {

  private final JLabel mLabel = new JLabel( "Hello, world" );
  private int mCount;

  public void init() {
    getContentPane().add( mLabel );

    setDefaultCloseOperation( EXIT_ON_CLOSE );
    setLocationRelativeTo( null );
    setAlwaysOnTop( true );
    pack();
    setVisible( true );
  }

  @Override
  public void nativeKeyPressed( final NativeKeyEvent e ) {
    final var s = getKeyText( e.getKeyCode() );
    System.out.print( s + " " + (++mCount % 10 == 0 ? "\n" : "") );

    invokeLater( () -> mLabel.setText( s ) );
  }

  public static void main( final String[] args ) throws NativeHookException {
    disableNativeHookLogger();
    registerNativeHook();

    final var harness = new Harness();
    addNativeKeyListener( harness );

    invokeLater( harness::init );
  }

  private static void disableNativeHookLogger() {
    final var logger = getLogger( GlobalScreen.class.getPackage().getName() );
    logger.setLevel( OFF );
    logger.setUseParentHandlers( false );
  }

  @Override
  public void nativeKeyReleased( final NativeKeyEvent e ) {}

  @Override
  public void nativeKeyTyped( final NativeKeyEvent e ) {}
}

The code above produces a small window that, when run, demonstrates the problem:

Be sure to type into any other window to see the perplexing loss of key presses within the demo application.

Environment

• OpenJDK version "14.0.1" 2020-04-14, 64-bit
• XFCE
• Arch Linux
• JNativeHook 2.1.0

Details

JNativeHook runs in its own thread, but using invokeLater (or invokeAndWait?) should issue the UI update on Swing's event thread.

The call to disableNativeHookLogger() isn't relevant, it merely keeps the console clean when running the demo.

Console Output

Here is the console output when the application has focus:

Shift I Space A M Space I N S I 
D E Space T H E Space A P P 
L I C A T I O N Period

Here is the console output when the application loses focus:

Shift I Space A M Space O U T S
I D E Space T H E Space A P
P L I C A T I O N Period 

So it's clear that no keyboard events are missing when nativeKeyPressed is called, regardless of whether the application has focus. That is, neither JNativeHook nor its event bubbling via JNI appears to be the culprit.

Question

What needs to change so that the JLabel text is updated for every key press regardless of whether the application has focus?

Ideas

Some items that help include:

• Call getDefaultToolkit().sync(); to flush the rendering pipeline explicitly.
• Call paintImmediately( getBounds() ) on the label.

The first item seems to make a huge difference, but some keys still appear to be missing (although it could be that I'm typing too quickly). It makes sense that preventing the rendering pipeline from merging paint requests avoids loss of key strokes.

Research

Resources related to this issue:

• https://pavelfatin.com/low-latency-painting-in-awt-and-swing/

Answer 1

Call sync() using the default toolkit:

  @Override
  public void propertyChange( final PropertyChangeEvent e ) {
    invokeLater(
        () -> {
          update( e );

          // Prevent collapsing multiple paint events.
          getDefaultToolkit().sync();
        }
    );
  }

See the full code.