JavaFX Minimizing & Maximing undecorated stage with animations

Question

I am using the accepted answer in this question: JavaFX Minimizing Undecorated Stage to minimize my app properly.

However, unfortunately the default Windows minimize & maximize animations are not shown at all (the window just appears and disappears).

I know it is possible to make the animations display with undecorated windows, since I have one application that has this behavior (PotPlayer).

How could I make the animations appear with JNA?

EDIT: Here's a working Kotlin code piece to minimize a JavaFX window properly, also added bounty.

fun makeMinimizable(stage: Stage) {
         val user32 = User32.INSTANCE
         val hWnd = user32.FindWindow(null, stage.title)
         val oldStyle = user32.GetWindowLong(hWnd, WinUser.GWL_STYLE)
         val newStyle = oldStyle or 0x00020000 // WS_MINIMIZEBOX
         user32.SetWindowLong(hWnd, WinUser.GWL_STYLE, newStyle)
    }

Answer 1

Upon further research on the Windows animations, it looks like a solution could be hacked together. It seems to be more of an OS issue and not just JavaFX.

I was able to get the initial window to stay undecorated while minimizing and with the animation by modifying this in start():

    int newStyle = oldStyle | 0x00020000 | 0x00C00000;

But, after minimizing and reopening, the Windows border appears oddly enough.

Then, I tried to use a ChangeListener to swap Windows styles when iconifying.

stage.iconifiedProperty().addListener(new ChangeListener<Boolean>() {

        @Override
        public void changed(ObservableValue<? extends Boolean> ov, Boolean t, Boolean t1) {
            if (t1.booleanValue() == true) {
                int newStyle = oldStyle | 0x00020000 | 0x00C00000;
                user32.SetWindowLong(hwnd, GWL_STYLE, newStyle);
            } else if (t1.booleanValue() == false) {
                int newStyle = oldStyle | 0x00020000;
                user32.SetWindowLong(hwnd, GWL_STYLE, newStyle);
            }
        }
    });

This successfully gets the windows un-minimize animation to work fine consistently, while leaving the (visible) stage borderless.

It looks like I can get minimization animations working once I find out the best way to re-apply:

    int newStyle = oldStyle | 0x00020000 | 0x00C00000;
    user32.SetWindowLong(hwnd, GWL_STYLE, newStyle);

just before the stage is iconified, and the border isn't visible to the user. Once implemented, this might work similarly to the C# solution in the first link below. Basically, what the above ChangeListener does in reverse.

I think we need another thread listening in the background to finish up this solution, waiting for the iconified event to happen. I think stage.isIconified() == false should be the case when the event is fired, and then we execute the necessary task in the background thread to set the above code. Then, the (...somewhat working) changeListener will reset it back to a non-framed Window without animations, when un-minimizing, until it is minimized again.

There is a minor bug with the first un-minimize displaying the bottom portion of my stage clipped and duplicated a little bit, but it disappears after subsequent actions. We may need to try changing the second hexadecimal to something else that triggers animations, and setting everything outside of the ChangeListener and in another thread.

I plan to finish this up soon in my own fx program. I'm still a student so I'm not too experienced with multithreading and services but I have a good idea on how to go about it, it will just take a few hours of hacking away.

Let me know if you make any progress in the meantime! This is a good starting point. Nobody has solved this issue in Java yet. I was reading up on some discussions that borderless programs like Steam have done this, but I don't think anyone could entirely figure out how they accomplished it, I doubt it was through the C# hack below. But, it's good enough for our purposes.

Links to do with solving borderless/undecorated animations:

https://stackoverflow.com/a/31489766/7234125

^ We need to implement this answer, option #1, from C# to Java

Borderless Window Using Areo Snap, Shadow, Minimize Animation, and Shake

http://pinvoke.net/default.aspx/Constants/Window%20styles.html