Disable Background drawing in JFrame in order to properly display Aero (DWM) effects

Question

I'm having problems using the DWM functionality of Windows Vista/7 on Java windows. I want to make the background of my frame use the Aero style. The Windows API to do so is provide by the function DwmExtendFrameIntoClientArea in the dwmapi library. I've managed to call the procedure properly via JNA, and it does what it is supposed to do (You can see that for example when resizing the frame, before the next repaint you see the proper aero effects in the area not yet painted, see the attached image).

But somewhere (I can't figure out where) a background is painted over the Aero effect and the effect is lost.

What I have already tried:

• Using a custom ContentPane with opacity set to false
• Setting the opacity of the LayeredPane and the RootPane to false
• Using a Frame instead of a JFrame
• Set the background color of the JFrame/ContentPane to black/fully transparent
• Use setLayersOpaque and a custom variant thereof, see first answer for more details

So far I could not succeed removing that background. Is it a limitation of AWT/Swing? How can I remove that background or use the Aero effect properly?

Your help is greatly appreciated.

Screenshot

Here a screenshot of a frame without any contents, having set the opacity of the RootPane, LayeredPane and ContentPane to false. I did it quickly while resizing. You see that the effect is properly applied to the area Java did not yet paint on.

http://i55.tinypic.com/v614qo.png (As a new user I cannot post the image directly...)

Odd behavior

Upon further investigation I came across the following odd behavior. If the window size is 150x150 or below the contents are displayed transparently. This is very glitchy for normal window components. If you paint directly on the frame by overriding the paint() method everything is drawn semi-transparent. Additionally the coordinate system seems to be a little off, it appears as the zero point of the JFrame is set to the actual zero point of the window. Thus Swing tries to paint to areas where actually the window border is located, which then of course is not visible.

See this screenshot: http://d-gfx.kognetwork.ch/java_aero_bug.png

Example code

This is the code I use.

Requires jna.jar and platform.jar. Available from the JNA homepage.

import com.sun.jna.Function;
import com.sun.jna.Native;
import com.sun.jna.NativeLibrary;
import com.sun.jna.Structure;
import com.sun.jna.platform.win32.WinDef.HWND;
import com.sun.jna.platform.win32.WinNT.HRESULT;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.UIManager;

public class AeroFrame extends JFrame {

    public AeroFrame() {
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        JLabel label = new JLabel("Testlabel");
        label.setOpaque(false);

        add(label);

        pack();

        enableAeroEffect();
    }

    private void enableAeroEffect() {
        NativeLibrary dwmapi = NativeLibrary.getInstance("dwmapi");
        HWND aeroFrameHWND = new HWND(Native.getWindowPointer(this));
        MARGINS margins = new MARGINS();
        margins.cxLeftWidth = -1;
        margins.cxRightWidth = -1;
        margins.cyBottomHeight = -1;
        margins.cyTopHeight = -1;
        //DwmExtendFrameIntoClientArea(HWND hWnd, MARGINS *pMarInset)
        //http://msdn.microsoft.com/en-us/library/aa969512%28v=VS.85%29.aspx
        Function extendFrameIntoClientArea = dwmapi.getFunction("DwmExtendFrameIntoClientArea");
        HRESULT result = (HRESULT) extendFrameIntoClientArea.invoke(HRESULT.class,
                new Object[] { aeroFrameHWND, margins});
        if(result.intValue()!=0)
            System.err.println("Call to DwmExtendFrameIntoClientArea failed.");
    }

    /**
     * http://msdn.microsoft.com/en-us/library/bb773244%28v=VS.85%29.aspx
     */
    public class MARGINS extends Structure implements Structure.ByReference {
            public int cxLeftWidth;
            public int cxRightWidth;
            public int cyTopHeight;
            public int cyBottomHeight;
    }

    public static void main(String[] args) {
        try {
            UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
            JFrame.setDefaultLookAndFeelDecorated(true);

        } catch (Exception e) {
            e.printStackTrace();
        }
        new AeroFrame().setVisible(true);
    }

}

Answer 1

Great question.

The most obvious answer would be

WindowUtils.setWindowOpaque(this, false);

That gives you the visual effects you want but unfortunately prevents you from being able to click on the Window!

The second thing I tried was to override the paint() method to perform the same actions that Window.paint() does when the opaque flag is set to false. That didn't do anything.

Then I tried using Reflection. Reflectively setting Window.opaque to true gave the same results as using WindowUtils.

Finally, I tried adding this to enableAeroEffect():

Method m = null;
try {
    m = Window.class.getDeclaredMethod("setLayersOpaque", Component.class, Boolean.TYPE);
    m.setAccessible(true);
    m.invoke(null, this, false);
} catch ( Exception e ) {
    //TODO: handle errors correctly
} finally {
    if ( m != null ) {
        m.setAccessible(false);
    }
}

This worked! The Window still responds properly to mouse events, but the background isn't drawn. The drawing is a bit glitchy, but should get you on your way.

Obviously it's fragile since it relies on Reflection. If I were you, I'd take a look at what Window.setLayersOpaque() does, and try to replicate that in a way that doesn't rely on Reflection.

Edit: On inspection of the setLayersOpaque method, it really seems to boil down to disabling double-buffering on the transparent components. Call this method from your enableAeroEffect() method and you're on your way:

//original source: Sun, java/awt/Window.java, setLayersOpaque(Component, boolean)
private static void setLayersTransparent(JFrame frame) {
    JRootPane root = frame.getRootPane();
    root.setOpaque(false);
    root.setDoubleBuffered(false);

    Container c = root.getContentPane();
    if (c instanceof JComponent) {
        JComponent content = (JComponent) c;
        content.setOpaque(false);
        content.setDoubleBuffered(false);
    }
    frame.setBackground(new Color(0, 0, 0, 0));
}